Qualitative researchers are often asked to supplement their research with quantitative methods, and for many, it’s easy to feel overwhelmed and intimidated by the prospect. However, quant and qual are more alike than you think!
While qualitative methods provide detail, color, and insights into users’ thought processes, quantitative research can bring a more scientific approach to the data. Sometimes quantitative methods can validate qualitative insights. On the flip side, quant can also be used to inform qualitative topics. All in all, quant can be very helpful to qual researchers looking to round out their research insights.
Here are a few pitfalls to avoid when working with quantitative methodologies.
1. Making assumptions
Some people think that quant research=surveys. But, quant is much more than just that. And, even within surveys, there are differences in how you can reach an audience, whether you’re working with panels, intercepts, or online communities. The fact is, quant is much more nuanced than just throwing a survey together. Be open to the different methodologies and approaches tailored to your specific research goals.
2. Exhausting participants
Participant fatigue is a big issue in quantitative research. When running a qualitative study, there’s a set amount of time a participant must agree to. It’s a conversation, and the commitment level is standardized. With quant, it’s much more nebulous. For example, it’s easy to underestimate how long a survey will take, and some participants might be faster than others. When developing your study, think about the types of questions you’re including and make sure you’re keeping it limited to a feasible number.
Here are a few tips to help you avoid exhausting participants:
- Limit the number of open-ended questions. With these, it’s more likely for participants to have a range of engagement levels, and for those who take them very seriously, it could mean extending a survey time by a lot.
- Matrix questions seem like only one question, but in reality, they’re many more. Think through how many individual questions your matrix is asking to make sure you’re keeping the survey manageable.
- Make your questions simple and easy to digest. This allows the reader to easily answer each question without stopping to read it over multiple times.
3. Jumping between topics
Arrange your questions in a logical order to help participants stay on track. The flow of questions is vital to making sure your participant is engaged, not confused by the topic order, and ultimately, giving accurate answers. Before finalizing anything, do a trial run as a mock participant, think about the flow of questions, and see how you do! When in doubt, have someone else review it to get a fresh perspective.
4. Asking leading questions
Just like with qual, leading questions don’t make for good quant research. You never want to make it obvious what you’re looking for. Explore creative ways to obscure the “right” answer for better accuracy, particularly in screening questions.
For example, instead of writing “Do you use [specific website]?” as a yes or no question, ask “Which of the following websites do you use?” and hide the correct answer amongst similar options. This prevents the participant from trying to guess what you want to hear.
5. Waiting to plan your analysis
Don’t wait to plan your analysis until after you run the research! Think about what you expect your results to look like, and determine which data points will be most valuable. In some cases, you’ll want to compare two segments and determine if there are any statistical differences between them. It’s important to frame your research plan with those segments in mind to ensure that you have the data you need for a compelling report.
Also, the myriad ways to analyze data may or may not be appropriate depending on the type of question you’re asking, the goals of the research, and the insights you’re looking for. It’s up to you as the researcher to determine which metrics make the most sense for your report and plan for those as you begin the research process. Don’t just default to what you know or what you always do, and remember, each question can be analyzed differently.
Have more questions about quantitative research and our quant practice? Let's talk.
