Last month, I had the privilege of attending Accenture’s International Women’s Day celebration which featured a keynote address delivered by Sheryl Sandberg about Lean In. The day was filled with inspiring stories from Sheryl, AnswerLab CEO Amy Buckner-Chowdhry, Stanford Professor Audrey MacLean, Fortune Senior Editor Pattie Sellers, to name a few, about how women find their paths to success and define it, as well as fascinating personal details about balancing life and family, while charging ahead with a career. Although the content was geared toward women, the four biggest takeaways for me from that day can be leveraged by both men and women.

1. Your career is a jungle gym and not a ladder: Sheryl Sandberg recounted a story about being offered a job at Google, but being hesitant to take it because there was no job clarity and little pay clarity. The Google CEO her told her “Don’t be an idiot Sheryl, if you are offered a seat on a rocket ship, you don’t ask what seat. You just get on.” Not only can we be more personally successful if we are more adventurous with our careers and look for high growth opportunities, but also our employers can be more successful if their workforces are filled with people who are unconstrained by traditional perceptions of a career ladder.
2. Joyfully participate in life: One of the panelists reminded me that work and life can be more fun if you set the framework of “joyfully participating.” To me, this means showing up to work every day ready to have fun and dive into opportunities rather than just taking what comes my way. I’m fortunate to have chosen a career path where I love what I do, but I’ve always separated work from the rest of my life. When you can step back and think of the whole shebang together and joyfully participate, work can be a lot more fun.
3. Get them to fly into the fog with you: Whenever I’ve had a big idea and want to motivate people to get on board, I’ve felt that I need to spell out every detail clearly and make sure everyone understands every aspect of the plan. Similarly, I’ve expected the same of my superiors whenever a plan is put in front of me. Audrey MacLean, a professor at Stanford and founder of many start-ups, shared that when she was starting companies there wasn’t always a clear cut plan and she had to find people who could fly into the fog with her. I will think twice about needing that detailed plan in the future.
4. This is your defining moment: Amy told a great story about a difficult professional moment earlier in her career. She approached her boss with what seemed like an insurmountable problem to seek her advice. Her boss responded that she, too, had a similar challenge early in her career and that this was simply Amy’s time to experience it. When presented with this perspective, Amy was able to make the experience less emotional and personal, and to see it as a career-defining moment rather than an insurmountable problem. I found that to be incredibly inspiring advice for handling massive challenges or even massive failures. Instead of letting them beat you down, having a mental model for thinking that the moment can define you can be extremely empowering.

After the event, Amy and I were inspired and continued the “Lean In” conversation and shared ideas at our next AnswerLab management team meeting. Our team was particularly taken with the concept of looking at a career path as a jungle gym and have used the concept in many conversations with our direct reports and peers.

It was valuable to have the opportunity to step away from the day-to-day details of work and have a day to contemplate my career and talk to others about defining success.

More about Accenture’s International Women’s Day, related research and research methodology:

Accenture’s annual celebration of International Women’s Day, now in its ninth year, focused on the theme: Defining success. Your way.

In November 2012, Accenture conducted an online survey of 4,100 business executives from medium to large organizations in 33 countries: Argentina, Australia, Austria, Belgium, Brazil, Canada, China, Denmark, Finland, France, Germany, India, Indonesia, Ireland, Italy, Japan, Malaysia, Mexico, Netherlands, Norway, Philippines, Poland, Saudi Arabia, Singapore, South Africa, South Korea, Spain, Sweden, Switzerland, Thailand, United Arab Emirates, United Kingdom and United States. Respondents were split evenly by gender and were balanced by age and level in their organizations. The margin of error for the total sample was approximately +/-2 percent. A full report on the research, “Defining Success,” containing info graphics, analysis of regional trends and other key highlights is available at www.accenture.com/IWD

Research methodologies must evolve constantly to keep up with today’s changing technology and user experience (UX) research is no exception. Regardless of the industry, platform or phase of product development, UX researchers must broaden their toolkits to include creative solutions that go beyond the standard go-to usability tests for design teams. The most successful product teams use different methodologies throughout the product life cycle, as each approach brings a different lens to the questions they are trying to answer and therefore generates unique insights.

User experience research is critical to ensuring overall success of an experience so I’d like to share a quick cheat sheet covering some of the most effective methods that you may not have tried, including what they are, when to use them and why you should.

Expert reviews

What: Seasoned UX researchers evaluate your designs or concepts against a set of heuristics (design best practices) and provide feedback and recommendations for improvements.

When: Great for low-fidelity concepts or prototypes to get insights early enough to impact key design decisions; also a great option if timelines don’t allow for user testing. In the example on the left, a design team was struggling with 10 potential design solutions on a mobile app. The expert review helped the team identify three clear front-runners that were further iterated and ultimately tested with users. This method is popular with design teams looking to optimize their designs prior to launch or to find out what is or isn’t working well with a current design.

Why: Research experts can identify many of the usability issues that would be uncovered in usability testing and can get you the insights more quickly than conducting primary research because there is no recruiting phase to the project – saving time (roughly two weeks) and money.

Wait, there’s more! Experts share examples of how others may have solved UX challenges, providing insight into best practices within and outside of your industry.

Online focus groups

What: A researcher leads a group discussion with participants online. The online tool enables participants to see and interact with each other and react to visual stimuli.

When: Focus groups are a great tool to understand attitudes and beliefs around concepts that could influence adoption. For example, if you wanted to explore the barriers and motivators to using mobile payments, a group discussion would yield a laundry list of use cases, opportunities and barriers. This method is popular with product managers or marketing professionals seeking strategic input into the product roadmap.

Why: Online focus groups have the added benefit of reaching a broad audience across a wider geographic range without requiring travel. With the ubiquity of Webcams, it is relatively easy to recruit most user groups for online participation and the benefit of participating in their own home means that participants are often more comfortable and therefore more forthcoming than they may be in a lab setting.

Wait, there’s more! Tools like online polling enable the researcher to pose a question to the group and elicit feedback privately before sharing with the group – capturing individual responses before having a wider discussion. A whiteboard feature also encourages interactivity. For observers, online chat (not visible to participants) helps create a “back room” experience so that you can share your observations and thoughts as a team while the research is unfolding.

Participatory design

What: Participants are led through a series of exercises whereby they use markers, sticky notes, labels and images to illustrate how they conceptualize information or processes.

When: If you really want to attempt to get inside the minds of your users, there really is no better way. In the example to the left, an automotive team was trying to understand how people think about the process of configuring a car and where they expect to find key calls to action. Interaction designers like this method because it helps them think through the architectural issues and overall information design of task flows.

Why: Especially helpful when designing process flows, participatory design can get tacit knowledge or beliefs out of the head of the user and onto paper. This is most critical if designing a new digital experience. It can help design teams understand the logical order of steps within a process and to know when and where to position calls to action. In short, it can help to avoid costly design mistakes before getting too far down the development path.

Wait, there’s more! The highly engaging format of these studies is exciting for both the participant and the observers; participants can think more freely with a blank slate in front of them instead of reacting to a more designed experience. In this way, more creative ideas often surface. Observers benefit from seeing the visual artifacts created and identifying patterns created by real users.

Diary studies

What: Participants self-report their activities and experiences with a product or service over a given period of time. Diary studies can incorporate a wide variety of participant feedback methods, including survey responses, photos, videos or short phone messages. The output is a set of data and artifacts that give visibility into people’s routines and habits.

When: Any time you need to learn more about the habits and routines and natural use cases for your products. They are also a useful tool to capture insights at specific phases during the user life cycle (e.g., a new or returning customer’s experience). Many product teams use the data collected from diary studies to form the basis of personas.

Why: Diary studies provide you with context as well as an opportunity to observe and discover details that don’t always come up in the lab. Diaries are particularly helpful in understanding mobile behavior, since the physical environment is such a big influencer on people’s activities and therefore can’t be observed in a traditional lab environment.

Wait, there’s more! The artifacts that are collected through diaries – including photos and stories – are powerful internally within the organization to help illustrate common behaviors and to give visibility to users’ environments. It helps to ground product teams in the real experiences of actual users as they work to evolve the product over time.

Intrigued? Read more in our recent blog post: Dear Diary: Uncovering 3 Details you Miss in the Lab.

Mobile surveys

What: A live pop-up intercept survey on your mobile site lets visitors evaluate their experience and deliver insights into needs, expectations, behaviors and site performance. This approach offers flexible, real-time feedback and survey results that can be integrated with non-survey data for cause-effect analysis (i.e., impact of a new design, advertising, promotions, etc.) and provide direction for ongoing enhancements.

When: Mobile surveys can be done on a continuous or pulse basis, depending upon the marketplace and competitive context. We recommend the continuous approach when the marketplace is dynamic and competitive; there is a need to associate in-market effects with the survey data; and there is heavy site traffic.

Why: Consumers today are visiting mobile sites for far more than just a quick check-in or update. They expect mobile sites to be fully functional, to quickly browse and search with limited scrolling and they expect assurance of security. It is crucial to quantitatively measure the user experience on a mobile site to understand behaviors on the channel, track KPIs, ensure the user experience is optimized and identify pain points and strategies for ongoing enhancements.

Wait, there’s more! The technology to measure the user experience on mobile sites is rapidly evolving, with more user-friendly, robust survey techniques. We can not only assess the site’s performance but do so in real time while understanding the tasks performed and relate to the actual situation in which the site is being used (i.e., at home, work, while shopping, etc.).

Intrigued? Read more: Best Practices for Quantitative User Research on Mobile Sites.

Remote mobile usability testing

What: Usability testing conducted remotely with participants on their own devices in the comfort of their own homes or offices. Observers can view remotely and watch everything the participant is doing on their mobile device.

When: Usability testing is a critical phase of a user-centered design process, as it is often one of the final stages before launching a new product. Usability testing is also sometimes used prior to a redesign to better understand what is currently working or not working on a live design. The unique aspect of this approach is that it is conducted remotely, on the participant’s own device. Note that this method requires a fairly tech-savvy research audience so that should be taken into consideration to ensure this aligns with your target users.

Why: While mobile usability testing can be readily conducted in a lab environment, the option to conduct testing remotely is important when you need to reach a low-incidence audience or when you explicitly need to reach users in distinct geographic markets. With remote mobile testing, you can reach users wherever they are.

Wait, there’s more! By conducting research remotely rather than in a lab, costs are generally reduced. This gives you the option to save the money or to invest the extra you would have spent on recruiting additional participants to get a larger sample size.

Eye tracking

What: Eye tracking enables us to capture the gaze of participants as they interact with a Web site while also tracking clickstream data to illustrate the visual attention and behaviors of site users. Heat maps (seen left) and gaze plots provide a visual map of “hot” areas of a site and also clearly show page elements/areas that are given limited or no visual attention.

When: Visual design teams use eye tracking when trying to precisely lay out page elements and to ensure the visibility of business-critical page elements (i.e., calls to action, purchase buttons, lead forms, etc.).

Why: This method can be eye-opening for teams trying to drive very specific and discrete user actions. For example, a client recently used eye tracking to optimize a lead form on its home page. We learned that a photograph that was co-located next to the form fields dominated the visual field, reducing lead form completion. After removing the distracting image, the form fields became the focus and leads immediately improved.

Wait, there’s more! The heat maps produced with eye tracking provide compelling visual evidence to a team about what is and isn’t being seen by the target audience. Additionally, eye tracking provides insight into common patterns for viewing the on-screen information via gaze plots, as well as first-clicked links and click paths through the site. This is especially useful in identifying unforeseen dead ends or opportunities to optimize navigation overall.

Continues to expand

As I’ve outlined above, the UX research toolkit continues to expand as new technologies and best practices evolve. I hope this cheat sheet inspires you to step out of your comfort zone and try something different the next time you embark on a UX study. Over time, a multimethod approach ensures you’re getting the most holistic view into the user experience you’re creating and helps deliver a great experience for your customers.

While the health-minded have been using mini devices to monitor daily movement (e.g. Fitbit) or track their runs (Nike+) for a while now, the latest mobile offerings track the sorts of things you typically have to go to a doctor’s office to keep an eye on. Some of these new devices measure vital signs and even test your blood sugar, all thanks to the fancy circuitry available in your average smartphone!

What can my smartphone really tell me?

For most of us, a disease that requires constant monitoring pretty well necessitates specialized equipment or many trips to the doctor’s office. Now, medical devices are harnessing smartphones to provide basic, and not-so-basic, lab testing capabilities from the comfort of home. Recent breakthroughs allow patients to use their smartphones to:

• measure blood pressure
• chart heart activity
• monitor blood sugar
• detect malaria

More gadgets are in the works, including one to test your urine. Meanwhile doctors are beginning to use cheap, lightweight ultrasounds and CellScope is marketing a product that turns your phone into a microscope! Need to see if your kid stuck a pea in his ear and you don’t have an otoscope handy? No problem – grab your iPhone!

How do smartphones enable mobile healthcare?

This mobile medical revolution takes advantage of many benefits of smartphones; these devices are:

1. Portable. Since many adults carry around smartphones anyway, harnessing these devices is a no-brainer for tests that require constant monitoring.
2. Configurable. Many smartphones have enough processing power to perform basic monitoring tasks, and the same phone can run several apps that each perform a different test.
3. Personal. Your test can be configured to you, and you can keep track of your history over time and run basic statistics on the data.
4.  Synched with your doctor. Web connectivity lets you send your results and measurement history directly to your doctor, which keeps all your care providers on the same page (and potentially alerts your doc if the numbers seem out of whack).
5.  Relatively inexpensive. While I’m not arguing that an $800 iPhone 5 is “cheap,” this cost pales in comparison to the cost of going to the doctor for constant, routine monitoring.

Good user experience is mission critical — lives are at stake!

As monitoring becomes routine for the untrained lay person, it is imperative that the technology is simple to use and interpret. Having intuitive user experiences is important for ensuring that data is obtained correctly and properly understood. A home monitor is worthless — and possibly harmful — if false positives send you to the doctor over and over, incurring needless worry and expense! And, in the opposite case, any problems that are detected must be clearly communicated to the user; these medical devices are actually cases where poor user experience design could potentially be life-threatening or life-saving. The FDA mandates and regulates human factors testing of mobile medical devices to ensure that designs are safe, effective, and efficient. However it’s up to us as designers and researchers to create wonderful, safe experiences for users and to ensure best practices in a rapidly evolving field.

To learn more about advances in mobile healthcare, check out an interesting new book on the subject – The Creative Destruction of Medicine: How the Digital Revolution Will Create Better Health Care (Dr. Eric Topol), and watch a recent “Rock Center” (NBC) report on the mobile revolution.

While we may not yet be at the level of a Star Trek tricorder to assess every possible condition at once, we’re working our way there. Are you using mobile medical devices? Share your good, or bad, experiences with us.

Given my background in digital user experience research and psychology, it’s perhaps no surprise that I was intrigued when I heard of Lumosity. Lumosity is an online brain fitness program that promises to exercise your brain, simply by playing a variety of online games for about 10 minutes a day. Its science-based parent company, Lumos Labs, strives to create innovative neuroscience-based tools that empower people to lead better lives through better brain health and performance. As a researcher, I’m always intrigued to experience science in action, especially if it promises a better life. Naturally, I immediately signed up.

How Lumosity works

Lumosity recommends you train about 3-5 days per week. Each training session includes 5 different games. Each of those games is focused on a different area of cognitive functioning: speed, memory, attention, flexibility, and problem solving. My competitive self was pretty excited to learn that each of the games is scored, which made for more exciting play and kept me coming back day after day (sometimes sooner, if I was chasing a new personal best!).

The games

The games are all quite different and fun (or frustrating) in their own way. My personal favorite is a game called ‘Word Bubbles Rising,’ in which you are given a four, three, or two letter stem and have 30 seconds to type as many words as possible that begin with those letters. For example, the stem might be ‘hap’. Go! Happy, hapless, haphazard, happen, happenstance, hap… The ‘Word Bubbles Rising’ game targets your flexibility and articulation. Not a bad skill to have.

The game that I find most challenging is ‘Raindrops’, which exercises your problem solving abilities with basic math equations. In this game, raindrops with arithmetic equations fall from the sky and you must solve the equation before the raindrop hits the ground. If three raindrops hit the ground before you solve the problem, the game is over. At first, the raindrops move slowly and contain easy equations, for instance 2+4. However, as you move through the various levels, the raindrops get faster, multiple equations come at you all at once, and they get more challenging. This game frustrates me, because it drives me crazy when I can’t quickly solve equations that I learned in elementary school.

Measuring progress

While you have fun and play these games every day, Lumosity continuously measures your Brain Performance Index (BPI) and your progress. These scores are broken down by an overall BPI, as well as subscores for speed, memory, attention, flexibility, and problem solving. Lumosity research claims you will see the greatest improvement in your BPI during the first 20-30 days of training. This was definitely true for me. After my first day of training, my overall BPI score was 572. After 20 days of regular training (4 days a week), it had improved to 699. After two months, it was at 737.

I look forward to continuing to exercise my brain and seeing how much it improves over the coming months. I learned this week that Lumosity also has an iPhone app, for training on the go, and I’m pretty stoked about it. Now I’ll be able to use that commute time wisely!

Impact

I’m curious if I will notice an impact in my day-to-day activities. Will my training help me remember names more easily? Will I be able to switch tasks more quickly? The jury is still out. Stay tuned.

I see the benefits. For example, the article mentions someone who earned $5,300 in a year letting others drive his Honda Civic worth $4,800 (though there are legal nuances depending on how much you earn). I think the idea of a sharing economy is fantastic as it pertains to things (such as extra space in one’s apartment, or car); I love that technology makes it possible to use what we already have more efficiently. There are some neat ways that science is harnessing technology to become more “crowd sourced” and allow lots of non-scientists to participate in meaningful ways (e.g. Galaxy Zoo, FoldIt, eteRNA). In the realm of eCommerce, eBay, who we might consider the original “sharing economy” company, states on their site that in 2011, the total value of goods sold on eBay was $68.6 billion. That’s a lot of extra income generated for eBay sellers!

I also see drawbacks. I have mixed feelings about the long-term economic consequences of our hyper-connectivity as it pertains to the sharing economy of “thinking work”. I’ve seen lots of friends (across all kinds of fields – programmers, lawyers, academic researchers, writers) take part-time or contract gigs long-term, though they would prefer more stable full time positions. On the one hand, it’s great that technology allows this to be possible – with a laptop and an internet connection you can do all kinds of work well with little overhead and at least make some money – but on the flip side, it can be a difficult way to make a living long-term for most people. Some examples of the downsides to the sharing economy were articulated well in a NY Times article from last June – Is Technology Fostering a Race to the Bottom?

In the end, I appreciate that technology allows stellar folks to rise to the top more easily than was perhaps possible before. Salman Khan of Khan Academy and Nate Silver of the FiveThirtyEight blog are examples of smart folks who started something, got noticed, and then went on to bigger positions. These days, if you have a good idea, you can just start working on it with off-the-shelf or cheap technology rather than waiting for someone else to fund you. The trick is to ensure that this ability to share and create benefits as many people as possible.

Technology has lowered barriers to entry across nearly every industry, which has both pros and cons depending on where you stand. I’m looking forward to continuing to track how the future of the sharing economy plays out.

That’s not what I’m seeing at AnswerLab. Focus Groups are back by popular demand.

Photograph: Mike Yarish / © AMC / Everett Collection

Mad Men may have had at least a small role in planting the seeds in our collective unconscious to give focus groups another shot despite prior years of waning interest. In past seasons, the show has featured several episodes with Dr. Faye Miller moderating groups on topics from lipstick to cold cream, giving visibility into those early days of market research and quite possibly sparking renewed interest among marketers and product managers. Skeptical? Consider the fact that the series inspired a “Mad Men Collection” at Banana Republic, enrollments in Advertising/Marketing are on the rise in universities across the country, and martinis are one of the most popular drinks on bar menus today.

Back then – just as today – the products that really hit the mark were those that “spoke” to consumers, and they did so by using natural language and referencing relatable experiences. Some widely recognized modern examples include Dove’s “Real Beauty” campaign, Levi’s Curve ID jeans, and Cotton, “The fabric of our lives.”

So why focus groups, and why now?

Of course, it would be an over simplification to suggest that a popular television drama could revive the methodology, and I’m not suggesting that Mad Men or SNL single-handedly brought “sexy” back to focus groups. What is more likely is that the rise of social media and the empowerment of consumers to engage in dialogues with large corporations have created a culture where consumers expect to be heard. Now more than ever, companies are venturing into market research to understand their customers – and focus groups are one of the most accessible and simplest ways to get close to the customer.

But just like the cocktail dress of another era must be modernized to fit into today’s fashion, so too must the focus group continue to evolve to be an effective research tool.

At AnswerLab, here are just some of the ways we ensure we get great insights from focus groups:

• Know when to use a focus group. Groups are best run in early-stage product development to understand unique use cases that the business should consider, and also to understand what barriers or motivators may exist that would either inhibit or support adoption of a product. Insights gathered at this phase should directly inform the product strategy. A great example would be understanding the potential for a “mobile wallet” – what does that concept mean to people? What problems would it solve? What new challenges might it introduce?

• Keep the group grounded in real-world behavior with “homework”. Current behavior is the best predictor of future behavior. Knowing this, one technique to facilitate a productive group is to include a homework component that asks participants to reflect on current habits and routines as a way to “prime” them for the activities they may do in the group. The facilitator can then refer back to these real world examples when asking participants to contemplate other scenarios. Using the mobile wallet example above, the participant’s homework may include questions about how they organize their wallets today, how they decide to use different payment methods, etc. The groups may then explore how their behaviors given a mobile wallet might remain the same. What might be different?

• Avoid “group think” by using a combination of individual and group exercises. One of the most common critiques of focus groups is the risk of losing the nuances of individual experiences in the muddiness of “group think,” where a dominant participant or two could sway an entire group to passively nod along with their point of view. Using exercises that individuals complete (either in advance as homework, or in the context of the group) ensures you capture the unique perspectives of each participant prior to discussing as a group.

• Don’t just talk – use role play or other interactive exercises to observe how people will act in “real” life. While dialog is obviously an important component of the focus group, don’t miss the opportunity to use techniques to get people out of their chairs and to role play scenarios you’re interested in exploring. For the mobile wallet groups, we created a café-like setting, complete with a cash register and a line of other “customers” in the form of life-sized cardboard cutouts of people. By creating this more realistic context, we learned how the social pressure of standing in line impacted expectations around speed and ease of use for a payment app that we likely would not have uncovered if we simply talked about mobile payments in the abstract.

For an outstanding modern-day example of these best practices in action, check out AnswerLab’s recent article about how PayPal leveraged these techniques in Quirk’s magazine.

While it’s not possible to draw a direct correlation between Don Draper’s use of focus groups to the recent interest we’re seeing at AnswerLab, it is nonetheless true that focus groups should be considered a classic and timeless research tool. Like the martini, there are ways to continually evolve and tweak it in the quest for perfection, but at its core is a solid formula that leads to enduring success.

For your viewing pleasure: Hidden Valley Ranch Taste test with Melissa McCarthy (Saturday Night Live)

The technology to assess a mobile site user experience has evolved substantially to allow for a more robust survey process. Recently, we have had a few opportunities to partner with Fortune 500 companies to help them evaluate their mobile sites. Executing quantitative UX research on a mobile site is significantly different from conducting traditional online surveys on a desktop computer. Here are some key best practices to foster success for your next project:

Prior to Launching the Survey

• Establish Limits. Restrict the scope of your research objectives since you will be employing a shorter survey that will be less expansive in terms of the number of metrics you can capture. If a shorter survey is not sufficient though, mobile site surveys offer the flexibility to have rotated / blocks of questions which only a subset of users respond to. The responses can be amalgamated together, but this approach requires a larger sample size for full sufficient sample sizes across questions.
• Know Your Site Traffic. Understand site- and page-specific traffic (number of unique vs. return visitors) to determine realistic sample and segment size goals, field length, whether an incentive might be needed, and designate the pages for the survey to “pop-up”. Web analytics data can help us provide reasonable expectations and predictions.
• Know Your Max. Establish clear sampling parameters up front in terms of maximum sample numbers allowed, especially if employing an incentive to limit the pay out in case an overwhelming response.
• Timing Matters. As with traditional sites, designate the sample rate (i.e. every n’th visitor receives a survey pop-up) and the time on the page needed prior to the survey initiating (5 to 10 seconds is a good starting place to allow the user to have some experience). Both these elements can be easily adjusted in field to facilitate achieving desired sample goals.

Introducing the Survey

• “Sell” Your Survey. Ensure the survey “pop-up” invitation on screen is unobtrusive and has branding consistent with the mobile site so that the survey looks and feels like part of the site visit. Employ a short, engaging invite that catches users’ attention and is honest about the survey length (e.g., “Congratulations! We would love to know your opinion about our site. Please take a moment to answer a very quick survey to help us improve our mobile site.”)
• Get the Right Device. Capture the device type in the survey (e.g., smartphone, tablet, MP3) and prevent those on a feature phone or desktop from taking the survey. This can be automated on the back end to ensure the only the correct respondents are captured.

Developing the Survey

• Short and Sweet. Limit the survey length to 5 to 7 minutes at most, since your respondents tend to be more time constrained and often on-the-go when taking a survey on their mobile devices.
• Keep It Simple. For an enhanced and engaging survey experience on a device that limits scrolling, use very simple and concise wording in questions and attribute text, restrict ratings scales to 5 to 7 points, have only one question per screen, and keep overall branding simple.
• Keep It (Mostly) Closed-Ended. Limit the number of “other specifies” and “open-end” questions to 2 to 3 to minimize typing on screen and the potential for errors and survey drop-off.
• Capture Context. Ask users where they are taking the survey and what tasks they are performing to capture valuable situational information and understand mobile site expectations and experiences.

Continuous vs. Pulse?

As with traditional desktop online surveys, the decision to institute a continuous versus pulse tracking methodology is dependent upon a host of contextual factors. Continuous should be employed when there is:

• Heavy site traffic to have sufficient survey responses over time. AnswerLab can work with you to determine the numbers needed and estimate the response rate and field time.
• A desire to relate in-market activities (e.g., advertising, news, promotions) to survey responses.
• A need to monitor trends before, during, and after ad campaigns/events (cause & effect) and reduce the impact of potential seasonality
• Capture unanticipated competitive and other activities.

Overall, mobile surveys are an extremely important, relatively lower cost, flexible, easy-to-implement, and valuable source of user feedback, offering real time data with dashboards. Brands that survey their mobile sites and use those insights to invest in and drive strategic decisions to enhance the user experience will develop a competitive advantage and ensure their brand is strong. Clients we have worked with have successfully used mobile site surveys to gain strategic insights into a host of areas, including determining how the site impacts their brand, the extent to which design changes have impacted the overall user experience, and identifying key drivers of call to action for ongoing enhancements.

If you want to learn more about measuring website user experience generally, check out our Guidelines for that. If you have general questions about when to apply which mobile UX research methods, see these Guidelines.

Of course, that sounds much simpler that it actually is. Here are five tips the panel shared for how to work successfully in an agile environment and focus on a common goal:

• UX research can guide teams towards the same goal. In an environment where changes have to be made on the fly, it’s easy to lose sight of the end goal and instead implement quick, easy solutions that work in the short term. But how will these solutions work for the users? Throughout the development process, UX research can help to answer this question and keep the entire team – product managers, designers, engineers – aligned on the same goal of “will this solve the user’s problem?”.
• Transparency is key to driving timely products. Agile is about responding to what is happening now; it’s about driving the future by developing products that match the needs of the present. In such environments, quick iterations are crucial and require full transparency about what everyone’s goals are. At Zynga, Aseron makes sure his research team’s goals are aligned with the rest of the business when embarking on a research project. He explained that when everyone’s goals are aligned, everyone can focus on working towards these goals, instead of focusing on the process.
• A research-oriented culture is crucial for Lean UX. Share findings within your company and let everyone see the value of research first-hand. Roberts encourages the entire Fitbit organization to watch research sessions by providing a streaming video link. Illustrating how studies can be impactful in working towards a common goal can increase support for further research. Always be an advocate for users and represent their voice through research.
• Be flexible. All of our panelists agreed that things change, especially in fast-paced, agile environments, and sometimes you need to be flexible and creative to reach the goal – and you need to do it fast! If you’ve ever played a sport, you probably know what I mean. You may have mapped out the shortest way to the goal, but if suddenly that defensive player shows up out of nowhere, you’ll quickly need to redirect your efforts to make it to the goal. Embrace change, even if the team’s goals have changed course, and think about how you can align your own goals with the new team goals.
• Get your questions answered and communicated in the most efficient way possible. Sometimes, an extensive ethnography study is the best and only way to answer your research questions and meet your goals. However, other times, you may just want to know if a button is discoverable; and you need to know so right now, before you can move on to something else. Use your already extensive tool kit to determine the best way to get the answer to your question. For example, when it’s a straight-up usability question, Lagrone walks over to another office and shows them the page. Do they get it? If not, it’s probably no good.

Our panelists were able to share great insights from years of driving UX practices in agile or lean environments, and their tips may be helpful in your own organizations. As we called out earlier, every company’s approach to agile or lean is different. That’s why it is extremely important that you work with your own UX toolkit. At AnswerLab, we work with many different clients and understand that everyone’s needs are different – what works for one organization may not work for another. You will need to determine what works best for your environment and how to implement lean UX in a way that makes the most sense for your organization.

When UX researchers use terms like “lean” and “agile” we are wading into murky waters because these terms mean different things to different teams and organizations. This post will NOT provide a prescribed formula for you to follow in order to implement “Lean UX” or fit research in an agile process. What it will provide are guidelines for successfully inserting user research into your agile or lean organization.

These guidelines are based on my experience as a researcher in lean organizations. In a former life, I wore the product and project manager hats at two different start-ups where I was responsible for implementing agile throughout the technology and product organization. More recently as the head of user research for a start-up, I had to figure out a way to match the existing lean and agile process of the product organization. This forced me to call on my agile roots and figure out how to elegantly fit UX research into leaner development processes.

First things first, realize the commonalities between the UX practice, lean and agile methodologies. Both agile and lean encourage building user-focused products. Isn’t this what we UX professionals have been preaching? Now it’s time to embrace and align our UX goals with these fundamentals.

Fundamentals of agile = collaboration, transparency, teamwork.

Ask yourself and your teams – how can we make research more collaborative, transparent, or better work into our teams?

• Collaboration. Invite people into your research. Have them help, take notes, or ask questions. Before your team joins you on this journey, teach them some best practices of research, such as listening and how to ask non-leading questions.

• Transparency. Show what you’re learning, even before final findings, and invite internal conversation. At a former company, my team would take over a wall and post our latest findings, we’d share learnings in the company newsletter, or proactively reach out to specific teams to tell them about what we learned. Of course, you want to remind your colleagues that you’re still collecting data, but taking them on the journey will substantially increase the likelihood that they include research findings into their product.

• Teamwork. Bring teams together to make sense of, learn from, and collaborate on the research. Once you’ve done all of this illuminating research, invite product team members, executives, and other stakeholders to join in a brainstorm or Design-Thinking sessions. At AnswerLab we’ve facilitated these sessions and have found that in addition to generating great ideas, it’s an excellent way to bring teams together and get them onto the same page.

Fundamentals of lean = constant feedback from customers, reduce waste.

Ask yourself and your teams – how can we get constant feedback and reduce waste?

• Constant feedback. You already know that you need to listen to your existing or potential customers, but are you operationally set up to have a constant stream of customers to talk to? Set your team up so that if they needed to, they could call someone in the next 10 minutes. Consider building a database of people who have opted in to speak to you.

• Reduce waste. Working in a lean environment, you need to get to the point fast. Three ways to do this are: get focused; iterate; and simplify your deliverables.

o Get focused. Need to get information to your team quickly? Perhaps reducing the scope of the project will help you get answers faster. Of course, doing so might only let you scratch the surface of the deeper learning. So, be sure you understand and explain the tradeoffs to the team. In my experience, the best time to get focused is when the team already understands the problem their product is solving, and therefore they are making incremental improvements to a UI.

o Iterate. Consider using techniques such as the RITE (Rapid Iterative Testing Evaluation) method. At AnswerLab, we’ve helped our lean-practicing clients conduct a round of sessions and then facilitated a debrief meeting to discuss our recommendations from the round of research. Then while the team is adjusting the prototype, we gear up for the next round of research sessions. Rinse and repeat as many times as the team needs to determine if they’re building the right product.

o Digestible deliverables. There’s a time and place for a comprehensive, detailed report. In a highly collaborative, transparent environment, this requirement is not needed as often. For AnswerLab’s clients that operate in this kind of environment, we typically facilitate a debrief meeting right after the last day of research sessions. In that debrief, we have all of the observers discuss what they saw. As a researcher, if we need to dig deeper, we can, but the team can take any immediate learnings and get to work.

By looking at the commonalties with UX practices, lean and agile can become your advocate, not a stumbling block to serving your users’ needs.

How are you practicing lean or agile UX? If you’re in the SF Bay Area and want to talk about ways research, lean and agile can play nicely, come to Thursday’s UXPA meetup panel discussion – Putting Lean UX into Practice this coming Thursday, 2/28 at 6:30pm.

Reinforce Our Core Values

Establishing and integrating our core values into our business processes has turned out to be our strongest recruiting and retention tool. We won’t make a hire unless we see examples of the candidate consistently exhibiting all six of our core values. To reiterate the importance of our values, we share employee core values stories in every company meeting, and we post them prominently in our offices. We’ve discovered that our team is most engaged when they work with people who have diverse backgrounds and talents, but who share the same values such as “build trust” and “figure it out and get it done.”

Leverage What Makes Us Unique

We try to create benefits and systems that are unique to us. Since AnswerLab helps companies understand what customers see, do, think and feel when using digital products, we aim to attract and retain employees who “embrace new technology” (another one of our core values). To reinforce this value, we established a tech benefit, where each year we reimburse every employee for purchases of new technology so that they can keep up to date with the latest gadgets, mobile phones, tablets and apps. Everyone on our team loves to learn, so we also established monthly learning lunches, where internal and external speakers share their knowledge. Finally, our office location is unique—it’s a block from the San Francisco Bay—so why stay locked up in the office? We established “walk and talks” to replace many of our one-on-one meetings.

Look for Triple Impact

We’ve found that our most successful programs have a triple impact— they benefit employees, clients and the company. For example, we started scheduling internal, quarterly “premier client meetings” to discuss how we can better serve our key clients and provide even more value to them. Everyone who touches the client attends. Team members like the chance to get to know each other in a different setting and enjoy taking part in the strategic-planning process. Conversely, clients appreciate the extra thought we put into how to serve them, and the end result is better service and client satisfaction. This creates a cycle of more employee engagement— happy clients equal happy employees, and vice versa.

People generally spend the majority of their waking hours at work. Creating a great place to work is highly satisfying for the whole team and it can have a profound impact on people’s lives.