Gain a Bird’s Eye View of the User Experience

Want to understand what the user experience is really like from the perspective of a customer? The data collected from web/app analytics, CES, VOC text analytics, video analytics, and session replays can show us where customer problems lie, but they don’t tell us why.

To inform your design decisions with deep, in-context insights, there are 5 different research methods you can use—depending on your research goal.

Tip: These same techniques also can be used to study competitor experiences.

Over-the-Shoulder UX Research Methods

Research GoalResearch Method
Gain first-hand knowledge and empathy.Service Safari
Track the user experience over time as a success metric.Mystery Shopper
Observe users' behaviors in real-time to capture attitudes & perceptions.Shop-Along
Observe users' behaviors in real-time to capture tasks & touchpoints.Contextual Inquiry
Gather users’ self-reported experiences over time.Diary Study

When to Use Which Method

Problem Discovery:
Service Safari
Shop-Along
Contextual Inquiry
Diary Study

Problem Validation:
Not applicable

Solution Discovery:
Not applicable

Solution Validation:
Diary Study

Performance Measurement:
Mystery Shopper

Service Safari

Research goal: Gain first-hand knowledge and empathy.

Also known as an “experience audit,” this research technique is not conducted with real customers or users, but rather by someone on the design team. The goal is to gain a first-hand perspective of the user experience, and then document the process, plus capture all observations into a hypothesis map or concept map.

Service Safaris are even more useful and insightful when conducted by multiple members of the team—so that you can gather and share differing points of view with each other.

Tip: Consider having new team members conduct their own service safari when they first join.

Mini Case Study:

When I was working on redesigning the website for a metropolitan public transportation organization—I had personally never ridden a public bus. Riding the bus from the suburbs to downtown for the first time allowed me to understand nuances of the rider experience (e.g., how to read bus stop signs, how to use the payment box, bus rider etiquette) and better inform my user research questions.

Mystery Shopper

Research goal: Track user experience over time as a success metric.

A mystery shopper, or secret shopper, is a contractor or real customer who is generally paid for their time (and given money for shopping, if applicable) to evaluate the user experience against a set of pre-determined criteria.

Example criteria might include: friendliness and timeliness of staff, cleanliness, the store or environment’s appearance, or specific merchandising / signage requirements.

Mystery shopping is usually conducted on a regular basis as part of the CX scorecard, and scores are tracked over time.

Shop-Along or Contextual Inquiry

Research goal: Observe the user’s behavior in real-time.

Shop-alongs and contextual inquiries are very similar research techniques. To decide which method is best, consider the desired outcomes of the research: do you want to learn customer attitudes or their processes?

A shop-along is just like it sounds: a researcher tags along with real customers on their shopping trip. The customer typically shops for a real product (as naturally and realistically as possible). Researchers document what they observe, and ask follow-up questions about the customer’s perceptions and attitudes about packaging, pricing, selection, merchandising, signage, etc.

A contextual inquiry is very similar to a shop-along, but has a slightly different goal and usage. Researchers study how users interact with the software, machine, or other user interface to complete their tasks. Additionally, researchers are focused on documenting and understanding every system, screen, and decision the user faces, their barriers, pain points, goals, etc. into a user flow or storyboard.

Can both methods be conducted simultaneously? Of course.

Diary Study

Research goal: Gather users’ self-reported experiences over time.

In a diary study, users document their experiences in a written, photo, or video “diary.” Researchers are able to gather insights more efficiently than real-time shop-alongs or contextual inquiries, but they lose the ability to ask follow-up questions in the moment.

Diary studies are especially useful when the research topic is sporadic (e.g., changing diapers), highly invasive (e.g., shaving), or extends over a long time period (e.g., days, weeks).

The outcome of a diary study often is a customer journey map.

Tip: Diary studies also can be used to evaluate the effectiveness of a prototype or proof of concept prior to launch—not just to study current state.

Tip: When faced with a tiny budget, consider using Google Docs to conduct your diary study. It’s free and accessible on phones and computers.

Conclusion

Use the above UX research methods in combination with other methods to develop a well-rounded view of the customer experience.

Related Articles

Author: Kristine Remer

June UX is led by Kristine Remer, a CX / UX research and strategy consultant in Minneapolis. She helps companies drive significant business outcomes by finding and solving customer problems. When she's not creating customer journey maps and user personas, Kristine is either kayaking or watching her kids play soccer.