Prioritizing Digital Inclusion: How to Recruit Participants with Low Tech Skills for UX Research
After a year of distance learning, remote work, and a major shift to online interactions, the digital divide has become ...
When the pressure’s on and timelines are tight, sometimes it’s hard to remember that qualitative user research means we’re humans speaking to other humans. In our quest to get the best data, we may forget this fact and treat our interaction as transactional. Or worse, make people feel as if they don’t belong or their responses aren’t worthy of our work. Inclusive research means not only that participants bring diverse backgrounds and ethnicities, but also that these underrepresented voices feel welcomed and their feedback is incorporated into product design decisions.
As researchers, there are several ways we begin to practice inclusive research: inclusive recruiting, conducting experience gap research, and creating a roadmap and framework for our research moving forward. In practice, we have an opportunity to welcome our underrepresented participants, treat them with respect, and include their points of view. We need to show up to our research sessions fully aware of the unconscious biases that risk alienating our participants. In doing so, we can see them as fully human and treat their responses equal to participants who exemplify the dominant culture.
There are many unconscious biases that people can exhibit. However, I’ve narrowed it down to five that may surface for moderators when interviewing people who do not present the dominant culture identity. Before you say, “but I’m not racist…,” it’s important to remember that the white supremacist dominant culture is so strong that sadly everyone has the potential to display racist thoughts and behaviors because of their unconscious biases.
Affinity Bias
Definition: Unconscious tendency to get along with others who are like us.
Make sure that you’re kind and professional with all participants.
Watch Out: In one session, you feel you really ‘get’ this participant. You like gaming, they like gaming, you’re getting along well. You write up your notes and feedback, highlighting many of their verbatim responses as possible quotes. The next participant, not so much. They’re a teenager who gives one word responses. You have nothing in common and their feedback is not as well articulated as the previous respondent. Your notes are minimal.
Act Inclusively: Regardless of whether you feel a connection with a participant or not, your work is to treat each of them equally. If a participant gives you answers or provides feedback that rubs you the wrong way - check in with that feeling and pause. This bias requires us to take more effort to act and bring more empathy to the conversation. Give the participant more empathy and sympathy for the participant who is different from you. Gently probe to get at their true sentiment. During reporting, pay extra attention to your selection of quotations. Pull quotes from all participants to illuminate your findings, not based on whether or not you liked the participant.
Confirmation Bias
Definition: The tendency to interpret new evidence as confirmation of one's existing beliefs or theories.
Ask the same questions of everyone in the same tone and manner so you collect data regardless of your opinion of them. Use caution with improvisation and don't share any harmful assumptions with the participant.
Watch Out: A participant speaks differently than you, so you may be unconsciously dismissing their feedback. Their accent makes it harder to understand, and it reminds you of a few jokes you’ve heard in the past. As you’re interviewing them, you think of these stereotypes and dismiss the value of their responses. You rush through a few questions and then skip over the follow ups because you can’t understand them clearly and are sure you won’t get the data you need from them.
Inclusive Research: Review the participant list beforehand, if there is information about their native language, account for a slower paced interview in your moderation guide so you can ask every question, get full answers, and take full notes. Video record the session so you can review the answers to get any data you missed during the session. The onus is on the moderator to find the data, not on the participant to give it.
Attribution Bias
Definition: The systematic errors made when people evaluate or try to find reasons for their own and others' behavior
Don’t discount a participant’s response if they express themselves in a way you don’t like.
Watch Out: A participant really does not like the prototype. The design team is watching the session and you are inwardly cringing at this feedback. You wonder how they're going to take it and hope their feelings aren’t hurt. The other participants haven’t been this offended by the new feature and design. You start to think, maybe the participant had a bad day, or maybe they have a chip on their shoulder, or maybe they’re just being dramatic because all people ‘like them’ are dramatic.
Inclusive Research: Capture and review the data equally for all participants regardless of why you think they may be giving the feedback they do. Your work as a moderator is to elicit responses that get to the “why” they feel the way they do. You take their “whys” and analyze the data, not come up with the “whys” for an individual’s response as a way to dismiss their feedback.
Halo Effect
Definition: A bias where we form an overall positive impression of a person and that influences how we feel and think about their character.
Do not promote or hold any participant’s feedback and observations as “better” than others because we find them more attractive or presenting an identity of the dominant culture.
Watch Out: The participant arrives. He’s a tall white man with a firm handshake and a deep voice. He makes you feel at ease and you begin the interview. You find yourself nodding along with his answers and probing into his answer with zealous curiosity. “What a great participant! Such a friendly guy!” you think afterwards. As a result, you highlight several of his statements as possible quotes for later reporting.
Inclusive Research: This bias can show up as a “natural” reaction to someone’s appearance, and you make a quick judgement about their character based on it. In inclusive research, be present enough to notice biased thoughts that come up in your mind, even if they are positive, so they don’t affect your impressions about the participant.
Horns Effect
Definition: A bias where we form an overall negative impression of a person and that influences how we feel and think about their character.
Do not downplay or dismiss feedback by participants who display a trait that you have negative feelings about.
Watch Out: The participant arrives for the session in a wheelchair and you start to panic. The whole session is about mobility and getting around. You think - they are in a wheelchair they cannot “get around” and are ready to dismiss them. You wonder if there’s a backup. You don’t want to mess up the study data with someone who clearly, in your mind, doesn’t know anything about the topic. You’ll have to talk to the recruiter about it.
Inclusive Research: Pause and ask yourself if the assumptions you’re making are really true. What facts do I have at that moment? The participant was vetted for their knowledge and met all the screening criteria to participate in the study. The Horns Effect bias is unconscious so we have to turn our consciousness to the belief we have in the moment and challenge our thoughts with the facts of the situation.