Project 7 of 8

Increasing trust in recommendations through personalisation

Convene is a Thomson Reuters product for conference information. The app shares information about attendees, speakers, and sessions at large professional events. A recommendation feature was proposed to facilitate networking and initiate new customer interest.

Completed at Thomson Reuters and City, University of London.

Tara
UX Designer
Myself
UX Researcher

Problem

Using recommendations to facilitate networking is uncommon in the professional space; many professionals prefer to network face to face or do their own research on other attendees. Recommendations needed to be persuasive and trustworthy.

Outcome

As part of my thesis research, I wrote the explanations which accompanied recommendations. In order to increase trust and persuasion, I matched the language of explanations to a user’s information need; specifically, their Need for Affect (response to emotional information) and Need for Cognition (response to numerical information). As a team, we prototyped the recommendations feature and I ran formative usability testing.

Objectives

1

To understand whether language use on LinkedIn is a good indicator of their Need for Affect or Need for Cognition (e.g. the degree to which a user responds to emotional or numerical information).

2

To understand whether trust and persuasion in recommendations increase if explanation language use is suited to user information needs (Need for Affect and Need for Cognition).

Objective 1

Survey and LinkedIn Screening 

To address the first objective, participants completed two scales to determine their Need for Affect and Need for Cognition. 

In the same survey they also consented to their LinkedIn profiles being screened for language use.

The sample needed to be large and varied to avoid false positive errors, e.g. where patterns between language use and information need are falsely detected because the sample is not generalisable. 

“I would prefer complex to simple problems”

Example statement from the Need for Cognition scale. 
Participants rated their agreement from 1 (extremely uncharacteristic of me) to 5 (extremely characteristic of me).

Thematic Analysis

To understand whether Need for Affect or Need for Cognition can predict language use:

  1. The highest and lowest scoring participants for Need for Affect and Need for Cognition were segmented into four groups. 
  2. Qualitative analysis was run against the language in their LinkedIn profiles (using the Linguistic Inquiry and Word Count; LIWC) via MAXQDA12.

“A committed psychology graduate with a passion for teaching and child clinical psychology…I am currently enjoying my role as a secondary school science teacher for Teach First, working towards a vision for a day where no child’s educational success is limited by their socioeconomic background.”

Statistical Analysis

Between-group comparisons were made based on self- reported Need for Affect and Need for Cognition, and LinkedIn word count according to the LIWC.

For example, the analysis showed that participants with higher Need for Cognition (more numerical information needs) were significantly less likely to use words in the ‘positive feelings’ linguistic category. 

These patterns were enough to suggest that language may also be interpreted differently by users. Therefore, we proceeded to objective 2. 

Objective 2

Prototyping

To address the second objective, the UX designer I worked with created some high fidelity prototypes and I inserted the recommendations and explanations.

As the information needs and other personality variables of participants varied greatly, this part of the study needed to be conducted within groups. 

I created two copies of the same recommendation, only differing by the language used in their explanations.

The biggest challenge was to avoid introducing design variables which may influence trust and persuasion. 

Formative Usability Testing 

Testing was run with a separate participant pool to objective one. Each session was conducted as a moderated think aloud session where participants reviewed recommendations and rated them on trust and persuasion. Using a think-aloud approach I was able to  explain or invalidate later ratings of trust or persuasion for each recommendation. For example, whether scores of trust were given based on the language or another reason. 

At the end of the session, the  participants also completed the Need for Affect and Need for Cognition scales.

“Passionately, as opposed to? What if he doesn’t work passionately, you can still be a bloody good lawyer or whatever can’t you, why passionately?”

Statistical Analysis

I tested for an interaction between explanation language use, participant Need for Affect and Need for Cognition, and the scores for trust and persuasion for each recommendation.

Although statistically significant, the results tended to suggest that using language which opposed a user’s Need for Affect or Need for Cognition could reduce trust. 

Impact and Results

This research was first of its kind to investigate

  • whether information needs are reflected in language use online,
  • and whether appealing to information needs by adapting explanations support trust in recommendations.

It initiated

  • the development of a recommender system using collaborative filtering (where recommendations were based on the users own interests and the interests of other similar users),
  • and further opportunities for the Applied Innovation Team to strengthen their position in the field of Cognitive Computing.

Upon reflection,

  • this project lacked the financial support to recruit a large and varied sample; only 112 participants took part overall. However, it was an important first step to gain interest in this kind of research in the organisation.
  • It confirmed my belief in the importance of user research when developing next-generation interfaces like AI where language use is hugely influential.