Having a baby changed everything about the way I conduct usability studies.
In the first 6 weeks after returning to work from my second maternity leave, I conducted 4 concurrent usability studies. Four sets of everything: Four sets of participants to recruit. Four sets of test materials. Four sets of client stakeholders.
Before that, I had usability documents here, usability files over there. I didn’t have a system and I was needlessly recreating test materials from scratch each time. It was terribly inefficient. And I was stresssssed.
No more. It was time to get serious.
So I set up a usability kit for myself, and called it “usability in a box.”
Inside my usability toolkit, you will find the following documents and templates:
Test Plan
Outlines the research goals, business goals, research and recruiting methodologies, tasks, participants, timing, observers, and deliverables.
Read more about how I structure my test plans.
Recruiting Invitation
Templates for emails, Facebook ads, Facebook posts, and Twitter posts to invite participants to the study.
Screener
List of questions to ensure the exact right participants are recruited. Including templates for explaining the study, copy for those who don’t qualify, and explanation copy detailing next steps for those who do.
Phone Scripts
Once the participant qualifies, I will call participants to check for articulation, verify their screener information, and schedule the session.
Lately, I’ve been using Calendly to have participants schedule a touch-base with me. Much more efficient, plus it quickly weeds out those who weren’t all that serious about participating in the first place.
Reminder Emails
Scripts for reminding participants about their upcoming sessions — plus online conference room instructions or driving directions. For in-person sessions, I’ll ask them to bring a photo ID and to arrive 10 minutes early.
Discussion Guide
The word-for-word script I use to explain the study to the participant during the actual session, plus ice breaker questions, tasks, and follow-up questions. Includes time stamps for each section to ensure I’m moving through the session at the right speed.
Participant Guide
The exact wording of a task is extremely deliberate. To avoid accidentally leading the participant, I type out the task — one task per sheet of paper — and have the participant read it themselves. Plus, it’s handy for them to reference again mid-task.
Observer Guide
This is a truncated version of the discussion guide I will be using. I provide this to help observers follow along and take notes.
Participant Schedule — for Observers
A list of the participants, their attributes (from the screener), and test schedule.
Participant Schedule — for Reception
Whether at a lab or at the client’s site, I’ll provide a list of the participants, their contact information, and schedule for the receptionist or greeter. To protect the participants’ privacy, I omit personal information, such as household income and other screener information.
Pre-Test Questionnaire
Sometimes clients want to collect a bit more information about the participant for more context about their background or past experiences. (This is usually a typing tool for segmentation purposes.)
Post-Test Questionnaire
A worksheet that I ask participants to complete following the conclusion of the test. This is especially helpful when testing the site or system over many years — you can show trends and progress when testing hundreds (or thousands) of participants over time.
Usability Log
This is just an Excel spreadsheet with columns labeled for issue, area (of page, site, app, etc.), issue type, participant #, and notes area.
During the test sessions, I’ll use Google Sheets so observers can collaboratively capture findings along with me.
Observer Guidelines
This is a standard outline of what a usability study is, what it is not, and specific instructions for collecting insights (e.g., observe facial expressions, see where they first click).
Plus, I include reminders to keep voices low during the study and not to laugh or make any outbursts. (One-way mirrors aren’t sound proof.)
I also reminder observers not to talk about the study outside of the observation room — especially in hallways or the restroom, where test participants might overhear.
Supply List
Everything I need to remember to bring or have ready for a day of testing. Refreshments, extra cords, paperwork, gratuity, clock, extra pens, clipboard, etc.
“Do Not Disturb” Sign
When conducting sessions at a client site, I do my best to ensure no one accidentally disturbs the test session. On the sign, I also include an arrow pointing the way to the observation room.
True story: 10 minutes before the end of a usability session, an unknown person came into the test room and wrote on the whiteboard: “I have this room in 10 minutes.”
Participant Consent Form
A brief explanation of the study along with a place for the participant to date and sign it. I have participants complete this and the pre-questionnaire in the waiting room.
Tech Troubleshooting Guide
Ah, technology. Technology makes my career possible, but things inevitably do go wrong in at least one session in every study. No need to panic, I have a list of solutions handy.
To Do Checklist
Ordering refreshments, securing gratuity, sending reminders, gathering supplies, reserving conferences rooms or lab, assigning a greeter for participants or late-arrival observers. Nothing gets left behind.
List of Customer Email Addresses
If I’ll be recruiting participants myself or hiring a vendor.
Prototype URL (+ Password)
I always have the prototype files bookmarked on the test device, but if the participant will be using their own device, then I create a document for the (shortened) URL for them to type or click.
If I’m testing a live site — then that’s part of the usability test — to see if participants have any issues typing the URL.
One Final Tip
With multiple versions of each file, this list of documents gets unwieldy quite quickly. To solve this, I organize the above into 3 folders:
- Before
- During
- After
Within the “After” folder, I have another folder called “Screen Caps” (abbreviated for screen captures). At the end of each test day, I take screen captures of every screen for the findings report — just in case the client team makes any changes.
Conclusion
Conducting the actual usability tests is a very small part of a usability study. There is an enormous amount of preparation to ensure a successful “show.”
Make it look easy, but maybe not too easy…
Many years ago, I overheard someone on the client team say to another team member, “I could do that,” referring to the usability sessions I had been facilitating that day.
In response, the other team member detailed everything I had done to prepare for the facilitation — the planning, the recruiting, the coordination.
“Oh, never mind. I don’t want to do all of that.”
[wink]
Related Articles
- The Art of Asking the Right Questions in Usability Studies
- 12 Less Obvious Usability Issues to Look For
- 3 Tips for More Successful Prototype Testing
- Your (Super Important) Role as a Usability Study Observer
- What to Usability Test First
