Over the past 15 years, I have conducted more than 400 usability tests on types of sites and apps. Here is my process.
My process for planning and conducting a usability study from start to finish includes 5 steps:
- Discovery
- Recruiting & Scheduling
- Script Development
- Facilitation
- Analysis & Findings
Below is a brief overview of each step.
1. Discovery: Laying the Groundwork
In this phase, I will learn about the objectives of your usability study: what you want to specifically learn and who you want to test your site or software. I use this information to write a usability test plan.
The test plan includes every detail that will be used for recruiting participants and developing the script, plus the testing methodology I will use and the overall schedule. It’s the foundation for everything that lies ahead.
There are several types of testing methodologies and hybrid approaches I use when conducting a usability study. One of the most common is called the “think-aloud methodology.”
Think-Aloud Method –
Simply, I ask the participant to “think aloud” as they complete each task. This inner dialog gives us insight into what is frustrating, interesting, confusing, and so on, so that I can understand where to explore more deeply. This information also gives me important clues as to how a usability issue might be best resolved.
But… I’m careful to not rely too much on what the participant says. What comes out of their mouth sometimes is not a true reflection of their experience.
This is why usability study facilitators are more interested in user behaviors rather than opinions (better suited for a focus group study) or preferences (better suited for A/B testing or an online quantitative study).
Once the test plan is approved, I move into recruiting and script development.
To learn more about the number and types of issues that I see in a typical usability study, please read my post Confusing, Incomplete Content Creates the Most Usability Issues.
2. Recruiting & Scheduling: Finding the Right People
The recruiting process makes or breaks the quality of any usability study. I use a rigorous approach to ensure I find participants who truly represent your customers or users.
The output from this phase includes 2 documents: the screener and the participant schedule. Working from the test plan, I develop a “screener” (a phone script or online survey) to help me find the right participants — and weed out those who are not a good match.
The aptly named “participant schedule” includes a list of the participants, the date and time of each session, and background information about each person (e.g., age, household income, profession).
Once the screener is approved, I move onto recruiting.
I find high-quality participants using one or more of the following methods:
- Your email database
- Your website via an intercept tool
- Your social media accounts
- An outside research panel
Depending on your target audience, I may look at alternative recruiting methods such as posting flyers at college campuses to recruit students, or recruiting your newest employees to test your company’s intranet or job board.
The amount of gratuity (the amount paid to the participant for their time) depends on the length of the session and the target audience. For example, the gratuity for a surgeon would be dramatically higher than for a college student. Cash always works, but I’ve found gratuity in the form of gift cards or high-value coupons also work well.
3. Script Development: Setting up Good Data Collection
In parallel to the recruiting and scheduling phase, I begin work on the script — also called a “discussion guide” or “facilitator’s guide.”
A comprehensive discussion guide ensures that each session is conducted in a consistent manner. That is, every participant completes the same tasks and answers the same baseline questions. I might rotate the tasks between sessions, or give different tasks to different types of users, but I always ensure there is a solid script to follow.
In a 60-minute usability session, there are typically 2 to 7 tasks that the participant will complete. Alternatively, I might create the tasks with the participant in the beginning of the session rather than have pre-defined tasks.
Each section of the discussion guide will include a timeframe to ensure the discussion stays on track.
In most cases, my discussion guides include 5 sections:
- Provide overview of session so participant knows what to expect.
- Ask ice-breaker questions to put participant at ease.
- Explain the methodology (i.e., ask participant to think aloud).
- Conduct the study (i.e., ask participant to complete tasks).
- Ask follow-up questions.
Once the discussion guide is finalized and participants are scheduled, we are ready for show time.
4. Facilitation: Uncovering Successes & Usability Issues
On testing day, the participant will be brought into the usability lab at a research facility or conference room at your location; or they will simply dial-in to a toll-free number (if it’s a remote study). As an observer, you will be able to watch the session from a computer. If the session is in-person, you will see both the participant’s face and the screen (but just their screen in a remote study).
The script is used to facilitate and guide the session, but the conversation can go in any number of directions based on what the participant encounters.
Some participants will only make it through one of the pre-determined tasks in the time allotted. Others will rush through all of the tasks at breakneck speed (I try to have additional tasks for these cases). And everything in-between.
Once a task is given to the participant, I try to interrupt them as little as possible. If the participant forgets to think aloud, I will nudge them with “What are you thinking about here?” But otherwise, I will mostly remain mute until the task is complete. I find this gives us better, truer insights into the user flow and experience.
Toward the end of each session, you and the other observers will have a chance to submit additional questions that I will pose to the participant.
In between every 2 to 3 test sessions, I will lead recap sessions with all of the observers. During this regroup, I will capture everything each observer saw and heard into a single spreadsheet. Recap sessions are critical to the success of a usability study. As a subject matter expert in your area, you will pick up cues from a participant that no one else will detect.
By the fifth session, the biggest usability issues will be absolutely clear. I recommend conducting a minimum of 8-10 sessions to uncover the majority of issues, but only 5 sessions are needed to find the biggest obstacles. Pretty cool, huh?
5. Analysis & Findings: Presenting the Themes
In the days following the usability study, I will conduct analysis of the findings, identify the usability themes, and then develop my recommended next steps and put them into a high-level presentation.
A typical report includes 6 sections:
- Overview of study (e.g., project background, test objectives)
- Usability study methodology used
- Profiles of participants and recruiting methodology used
- Executive summary of high-level wins and opportunities
- Detailed findings and recommendations
- Appendix of all documentation (e.g., test plan, session recordings)
I like to deliver the presentation in person as well as allow enough time for a meaningful discussion at the end.
I know presentation decks are often shared with teams who were not involved in the usability study, so I ensure the report provides the background and context needed to be immediately actionable.
If you need additional help solving critical usability issues, June UX also provides digital strategy, user research, user experience design, and content strategy services. Need more information? I’d love to chat with you.
Related Articles
How I Remote Mobility Usability Test
How I Plan User Research
