A global technology client asked us:
Does how we currently organize our APIs match the mental models of our developer users?
A global tech client approached us for help understanding the mental models of a niche segment of their users: developers. They wanted to ensure that the way their APIs are organized on their website aligned with developer expectations and thought processes.
We answered:
AnswerLab recommended a three-phase approach to deliver an in-depth analysis with targeted next steps for our client.
Qualitative IDIs and Card Sort
First, we conducted an initial phase of qualitative research to uncover how developers define, organize, and structure information and evaluate current terminology and product information. We conducted in-depth interviews with 15 participants across multiple global markets, including a qualitative card sort and review of a proposed information architecture (IA). Participants included software engineers, data scientists, product roles, and more. Through this phase, we surfaced several high-level themes around terminology, types of categories and tagging systems, and what participants valued in their daily tasks.
Information Architecture Refinement Workshop
Next, our researchers led a workshop with client stakeholders to present the findings from Phase 1 and refine the card sort exercise for the final phase of research, a Quantitative Card Sort. This workshopping process involved removing unnecessary cards, renaming categories and titles based on the naming convention research from Phase 1, merging cards with similar meanings, and more. As a result, the stakeholders left with a deeper understanding of their audience and a new and improved card sort to test in a quantitative phase.
Quantitative Card Sort
Lastly, we led a quantitative card sort to validate and quantify the findings from the initial qualitative phase and test the newly refined card sort from the workshop. In this phase, we led an asynchronous online survey with 200 participants at multinational companies, ranging in responsibilities from product to engineering to data analysis roles.
During the card sort, participants were presented with a list of cards and asked to sort them into pre-defined categories based on where they felt they should sit within the information architecture.
Outcome:
Through this study, AnswerLab led our client through a multi-phased journey to help them reach a data-driven decision around their website’s information architecture. Not only did we offer them qualitative findings on mental models, but we also helped them refine a proposed IA and re-test in a quantitative setting to validate the decisions. This gave our client team the confidence to make the proposed changes on their website armed with data from real users.
Our research deliverables included:
- A detailed report outlining their user base’s thoughts, needs, and wants when it comes to APIs on their site
- Qualitative card sort results broken down by each participant
- A recap of workshop findings and discussions for future reflection
- Quantitative card sort matrix, as well as visualizations showing participant reflections from the survey on confidence and difficulty levels
- Targeted recommendations for Information Architecture improvements
...
Mixed methods research offers a comprehensive understanding of user interactions, from the depth and nuance of qualitative findings to the validation you get from the big-picture quantitative data. This enables you to make more informed and data-driven decisions.
Explore another example of mixed methods at work when researching hardware.
