Too often, under-represented groups do not have great digital experiences because they’ve been historically excluded by outdated research and design processes. It’s time to change that. We recently hosted a webinar with AnswerLab’s Gina Tenenbaum and David Muñoz to share how AnswerLab re-examined and reformed dated research operations processes to create more inclusive recruiting for research.
We had so many thoughtful questions on this topic from our attendees, we couldn’t cover them all during the event. To help you take a more inclusive approach, we asked our panelists to help respond to some of the most common questions we heard to continue the conversation.
In international markets, we rely on our local vendors to help us understand the local cultural context. No one person can be an expert in every country’s culture, so we rely on those who are there on the ground. To help our local vendors understand why an inclusive approach is important, we share our philosophy and standardized US questions and choices. We then work with them to alter these to fit with local norms and include appropriate race and ethnicity options. We also make sure that the screener and any prototypes, moderator guides, etc., all translate correctly with an inclusive lens. If you’re conducting international research and want to be inclusive, make sure you’re speaking to local experts who understand what participants may expect or want from your screeners.
As with everything we do, we communicate that our findings are based on a small sample size. While we want everyone’s voice to be heard, each participant is still one person and their lived experience can vary greatly from someone with the same race or ethnic background.
To avoid tokenizing participants, we recommend you do the following:
In our work, we communicate the why behind our inclusivity focus by sharing materials, articles, and UX case studies for how this makes a difference. We’ve found most of our clients are not only open to the idea but also excited to get started. Often, after an initial study using this framework, our clients are so impressed by what their participants bring to the conversation and the findings they unravel, they want to make this a standard across studies.
We also recommend encouraging stakeholders to attend multiple sessions (or watch as many recordings as possible) so they can see the range of opinions and experiences that your participants provide. Seeing that diversity will also help your stakeholders see the value of all participants, not just those that they consider perfect.
Review participant criteria carefully with your stakeholders, asking probing questions about different aspects or requirements as you go. By asking why, you might discover your perfect participant comes from outdated preconceptions of who is using your product, and as a result, you can expand your criteria in your screener.
We understand some questions can be sensitive, such as race/ethnicity and identifying with LGBTQIA+. We want our participants to be comfortable throughout the process and instill a sense of trust wherever we can. For those two questions, we always provide “Other” and “Prefer not to Answer” as answer choices for those who don’t want to give us that information.
In our tracking system, we do not include any names or other PII, just the 6 demographic data points. When we look at reporting and analyses, we only see those 6 metrics on the following levels: by study, by each client, and across all studies we conduct. Because we remove any PII that identifies our participants with these demographic points, we don’t feel we are ever compromising sensitive or private information.
To start, we selected a few key metrics and have been optimizing those first. In the future, we may add metrics such as additional races/ethnicities. Black and Latinx people are historically underrepresented in the U.S. not just in terms of research participation, but also in technology jobs in general. In the United States, Black and Latinx workers account for 7% and 8% respectively of the tech industry, even though they make up 13% and 18% of the U.S. population. This is why we chose to focus on Black and Latinx participants for our initial inclusivity success metrics.
However, this doesn’t mean we aren’t including other populations in our studies. We do include Asian Americans in our studies but do not currently require it. This is because the percentage of the population (5.4%) does not equate to one whole participant until we get to n=24 and most of our studies have a smaller sample size than that. We are looking across many clients and verticals though, so certain companies with a larger Asian American representation in their user group might want to emphasize that metric first.
And lastly, this is also where our tracking system is supporting our on-going accountability and process iteration. We are watching our metrics across different groups over time to see where we’re doing well and where we can do better. Even without standardized quotas, it helps us monitor and adjust what we’re doing to ensure we’re not leaving anyone out.
To start, we recommend summarizing any quantitative data (site analytics, survey data, etc) that shows who your user base is and subsequently calling out how well (or poorly) your research studies currently match those analytics. Your product most likely has many diverse users. If your research doesn’t reflect that diversity, your product is likely not addressing all of your customers’ needs. If your competitors are, then you won’t be able to keep up.
To continue building your case, call out findings that you wouldn’t have learned without speaking to more diverse participants. For example, feedback from a 60-year old participant that the font size and colors used in a design are too difficult to read likely helps all your users and creates a better experience for older adults. Keeping a running list of these findings and their impact can help show the value of bringing in diverse participants.
Reaching participants who lack access to technology or WiFi can be tricky, but these participants can provide meaningful perspectives on a range of products and services. We recommend a few strategies to reach these participants, including:
Download your copy of our guide on Inclusive Research Operations to learn more.