participant bias

Does Anyone Really Like My Stuff? Testing the Effects of Participant Bias in Evaluation

Posted by AnneryanHeatwole on Jan 23, 2012

Many mobile and tech for development projects elicit enthusiastic responses by the target constituency when they are asked, but then go on to not be used. "Yours is Better!" Participant Bias in HCI investigates the influence of researchers and developers on how beneficiaries react to new services and products, specifically the social and behavioral reasons why users may claim to like a project or find it useful. In an effort to understand the relationship between interviewers and interviewees, two researchers ran an experiment to test the effects of participant bias. Covering interviewers with 450 residents of Bangalore, India, the experiment tested three hypotheses:

Does Anyone Really Like My Stuff? Testing the Effects of Participant Bias in Evaluation data sheet 4745 Views
Countries: India

"Yours is Better" Participant Bias in HCI

Posted by AnneryanHeatwole on Jan 20, 2012
"Yours is Better" Participant Bias in HCI data sheet 624 Views
Nicola Dell, Vidya Vaidyanathan, Indrani Medhi, Edward Cutrell, William Thies
Publication Date: 
May 2012
Publication Type: 
Report/White paper

Although HCI [human computer interaction] researchers and practitioners frequently work with groups of people that differ significantly from themselves, little attention has been paid to the effects these differences have on the evaluation of HCI systems. Via 450 interviews in Bangalore, India, we measure participant response bias due to interviewer demand characteristics and the role of social and demographic factors in influencing that bias. We find that respondents are about 2.5x more likely to prefer a technological artifact they believe to be developed by the interviewer, even when the alternative is identical.

When the interviewer is a foreign researcher requiring a translator, the bias towards the interviewer’s artifact increases to 5x. In fact, the interviewer’s artifact is preferred even when it is degraded to be obviously inferior to the alternative.

We conclude that participant response bias should receive more attention within the CHI community, especially when designing for underprivileged populations.