3 Remote Research Challenges Our Team Solved This Spring

Remote research has been a core methodology in AnswerLab's research toolkit for years, but as COVID-19 halted in-person research this spring, our team jumped into action to take our remote practice to the next level. As the needs of our clients and participants evolved, we evaluated new tools and practices, and refined solutions that helped us deliver impactful insights while overcoming a new set of obstacles.

While some aspects of research afforded basic transitions to remote methods, others have required more creative approaches to convert. Today, we’d like to share three big challenges we discovered this spring and how our team solved them, from the tools and platforms we employed to new processes we implemented:

Challenge:

We started to see stakeholders lose focus or stop attending sessions altogether.

Many of our clients found attending and observing in-person research together with their colleagues was an invaluable part of the research process. Excited to hear directly from their users, our stakeholders would watch sessions live, just a room away. They were energized by being in the room with their colleagues, which allowed them to talk through observations throughout the day, jot down notes on post-its, and brainstorm next steps all together.

However, when we moved to remote methods, we saw some of that engagement start to slip. It was harder to share notes and connect while co-watching participant sessions from different locations, and some of our stakeholders lost focus or stopped attending sessions altogether.

Solution:

Give observers a tangible way to engage.

To preserve the energy and value of the in-person experience, we had to innovate, so we dove in to find the best tools and platforms for the job. We found that supplying stakeholders with the right tools and prompts gave them a tangible way to connect with the research and with one another. It revived the energy of the backroom. Here’s what we recommend:

Use Mural for virtual collaboration

Back when we were conducting in-person research, we had a client team who came to our lab to watch sessions live every week. In the backroom, they liked to tack up the primary research questions on the wall, and then during each session, they would pin up their observations about each participant. The result was a visual depiction of patterns they noticed throughout the day. To get this same effect online, we knew we’d have to find a way to do this exercise remotely. Enter Mural.

Mural provides a virtual workspace that enables teams to take notes, capture observations, annotate screenshots, and ultimately, build alignment. Mural’s features include easy voting on notes and observations, sketching or drawing out your ideas, using virtual post-it notes, and exporting your work after the study is complete. With the help of this platform, our clients have a space to think critically about participant reactions and quotes, discover key themes, and collaborate on next steps.

Observe on UserZoom GO's research-centric video

It’s fairly straightforward to use any standard video call platform for remote research, but we wanted to explore options that could improve stakeholder engagement. Unlike typical conferencing solutions, UserZoom GO (formerly Validately) is a research-centric tool that allows observers to join anonymously and silently, and features increased security and confidentiality, advanced notetaking capabilities, and quick access to session recordings. In UserZoom Go, observers (with "collaborator" access) can take time stamped notes, tag themes, and communicate directly with the moderator in the app without the risk of accidentally messaging the participant. It keeps things cleaner and more streamlined for easy observing. 

Build alignment with end of day communication

Even if our clients can’t watch sessions live, we keep them engaged with the research through a new formalized end of day debrief process. We’ve found that touching base at the end of the day to connect on session progress and summarize key findings can help keep stakeholders engaged. This touchpoint ensures your stakeholders are learning quickly and getting fast feedback, regardless of whether they attended sessions. And as a bonus, it often keeps enthusiasm high and encourages stakeholders to tune in for the next day of research!

Read more about how we create our virtual backroom

Challenge:

We needed to facilitate group interactions remotely that had been envisioned as in-person.

At the start of shelter-in-place orders, we had plans for participant group interviews and stakeholder workshops that relied on being in a physical space together. Our researchers wanted to tack post-its on the walls, facilitate dialogue amongst colleagues, and use in-person dynamics to keep the project moving forward. As in-person interactions became impossible, we challenged ourselves to implement virtual solutions instead of postponing or cancelling altogether.

Solution:

Use platforms that simulate gathering “in a room” and invest in advance planning.

When gathering individuals “in a room” to synthesize research or conduct brainstorming activities, we’ve discovered certain platforms and advance planning can make these efforts a success remotely.

When we transitioned to remote research, we had a set of in-person workshops planned for a team of stakeholders from across the country to ideate on their research progress. The goal was to take a set of personas developed in a previous quant phase of research and develop empathy maps for each one. The last thing we wanted was to let their enthusiasm and excitement fizzle out just because we could no longer be in person. Our challenge was to keep them engaged and participating, outline productive next steps, and find a platform to host the empathy mapping exercise.

Make adjustments

To ensure success, we made some simple logistical changes, including shortening the overall time, planning short breaks throughout, and sending out the agenda in advance. We also split the larger group into smaller breakout rooms throughout the day to make sure everyone had the chance to speak.

Invest in adequate prep time

Again, we used Mural to help simulate in-person interactions and collaboration opportunities. Obviously, there’s always a learning curve that exists with a new tool, so we assigned a dedicated “scribe” for each group in advance and hosted a pre-workshop Mural demo to ensure they felt prepared. Taking time to prepare ahead of your workshops is always valuable, but in remote settings, it can be even more critical to ensure a smooth and seamless day-of experience.

Avoid unstructured gaps

In in-person interactions, the moderator or facilitator gets a constant signal about the engagement and energy of the attendees, just by picking up visual and auditory cues from what's going on in the room. In remote environments, it can be difficult or impossible for the moderator to intuit in this way. Plus, once an attendee wraps up an independent activity in a virtual workshop but before the moderator regains command of the group, the internet is at their fingertips waiting to pull them into a rabbit hole, breaking their focus on the session. One of the ways we mitigate this is to focus on group activities in remote sessions, rather than independent work. If independent work is necessary, we try to ask attendees to complete these as pre-session assignments.

Challenge:

We were asked to conduct research with populations we had previously assumed would be too difficult to recruit and/or interview remotely. 

Transitioning to remote research with certain populations can sometimes severely impact the experience of everyone involved: participants, stakeholders, and researchers. Accessibility research, for example, can be very difficult to do remotely. Reaching low-tech markets in a remote setting can also be tricky due to participant abilities, device constraints, and WiFi concerns.

Solution:

Responding with flexibility, conducting walk-ins, and changing recruiting strategies can mitigate these concerns.

While we haven’t found solutions for all of these challenges, we’ve discovered a few ways to make remote sessions in these situations more likely to be successful.

Recruit pairs and family members

In some cases, we’ve begun recruiting pairs or family members, so someone is available to assist the primary participant throughout the session as needed. Having someone in the room can make a big difference when trying to troubleshoot technical difficulties.

Conduct remote walk-ins

We always conduct remote walk-ins to help ensure tech issues don’t compromise session quality or eat up too much valuable time. Our team calls participants just prior to the session to help walk them through the process of joining the video call. This minimizes tech hurdles and ensures participants feel mentally and physically prepared to join.

Be flexible

While we do whatever we can in advance to make sure technical issues don’t arise, researchers need to be flexible and prepared to help participants troubleshoot. This can be frustrating in the moment, but don’t let it derail you. Respond with empathy and employ your best problem-solving skills to get things back on track.

Remember, not everything is possible

We recently assessed the feasibility of group research in a developing nation with participants who were blind or low vision and deaf or hard of hearing, as well as groups with the caregivers of those same participants.

In that country, individuals with disabilities often don’t have the means, access, or infrastructure to seamlessly connect via remote methods. Some participants might be able to connect via audio, but not video, or vice versa. A remote connection, with likely some technical issues and an added layer of cultural considerations, wouldn't encourage the level of sharing needed for the research, especially for a discussion we thought could get emotional.

This served as a good reminder that for some types of research, waiting for an in-person opportunity may be the ultimate preference after weighing the benefits and costs of going remote. While there are numerous ways to mitigate these concerns, there were significant trade offs in this case. We’re still exploring how we can conduct these kinds of research remotely, while maintaining quality and comfort for all involved.

...

We’re always thinking outside the box to overcome challenges and obstacles in the research process. It can be easy to get bogged down by technical or security constraints, but with the right tools and practices, we’ve found even some of the most complex methods can translate to effective remote research. 

Want to learn more about our remote research solutions? Get in touch with our remote research experts!