Facilitating a usability study is a lot like juggling.<\/p>\n\n\n\n
Not only do you need to pay attention to what’s happening on the screen in front of you, but also:<\/p>\n\n\n\n\n\n\n
Becoming a confident, highly skilled usability study\u00a0facilitator takes lots of practice.<\/p>\n
Here are my 10 best tips:<\/p>\n
Leading questions\u2014questions that subtlety or overtly tell the participant the \u201cright\u201d answer\u2014prevent deeper understanding.<\/p>\n
Not asking leading questions is one of the most difficult aspects of testing. It’s important to pause and think about how to carefully word unscripted follow-up questions.<\/p>\n
To avoid accidentally asking a leading question, I use a standard set of follow-up questions after each task, such as:<\/p>\n
These types of\u00a0questions help me understand where the site or app might not be aligned to the user\u2019s expectations. (Is the most important content on the bottom<\/em> of the page? Is key information missing? Are they seeing\u00a0what the business team wants them to notice?)<\/p>\n By calling an element a \u201csearch\u201d button or \u201cvideo\u201d or even just \u201cbutton\u201d \u2014 you miss the chance to identify whether or not the user understood the element\u2019s purpose, functionality, or meaning.<\/p>\n If you point to the magnifying glass icon, then say, \u201cIs it clear that this is a search button,\u201d an opportunity is missed. It’s better to ask:<\/p>\n If I don’t feel confident about an area I’m testing but haven’t yet learned everything I need to, next I will try:<\/p>\n In my opinion, a usability study that uses\u00a0leading tasks is a waste of time and money. A usability study is already somewhat artificial\u2014having someone use a website while a room full of people watch is pretty unnatural. Telling people what to do pushes it over the line.<\/p>\n For real learning, tasks should always be written in a way that forces the participant to THINK\u00a0about the problem they\u2019re trying to solve.<\/p>\n For example, instead of telling participants which exact links they should click, say, \u201cIf you wanted to look for more information about [topic], how would you go about doing that?\u201d<\/p>\n \u201cGo to the furniture section of the site, then show me how you would place a sofa in your shopping cart.\u201d<\/p>\n The participant selects the first sofa they see, then adds it to their cart.<\/p>\n What did you learn? Not much.<\/p>\n Here\u2019s what you might miss out on:<\/p>\n \u201cYou would like to replace your existing sofa. Show me how you would find the perfect one.\u201d<\/p>\n This type of task is more realistic and will yield many more useful insights.<\/p>\n It forces the participant to imagine a solution that will realistically work for their taste, budget, needs, and space limitations. It reveals what barriers exist and other unforeseen influences in their decision-making process.<\/p>\n In your tasks and follow-up questions, don’t use the labels that are on the site or app.<\/p>\n Rather than, \u201cShow me how you would return to the home page,\u201d when there is a HOME link staring them in the face\u2014instead try, \u201cHow would you return to the front page of the site?\u201d<\/p>\n Sometimes for clarification, it’s necessary to call something what it is called on the site. When that is the case, I will arrange the tasks in an order that exhausts studying the usability of the thing, before I start calling it by its name. Is it the right label? Is it noticeable and understood? We cannot know for sure unless we test it.<\/p>\n To get around calling something what it’s labeled:<\/p>\n Also avoid calling elements an icon, button, or link. Don’t point to the envelope icon, then ask if they understood the \u201cemail\u201d feature. Nor inadvertently call something a link\u2014the participant\u00a0may not have known that text was clickable.<\/p>\n Also, if your participant keeps mispronouncing something on the site, or refers to an element as something else\u2014don’t correct them. Just go with the flow and use it, too, if possible. It doesn’t help anyone to make a participant feel dumb while everyone is watching.<\/p>\n For example, I once had a phone interview participant refer to LinkedIn as \u201cLinkadink\u201d several times. I just went with it and called it that, too. (And actually was fun to say!)<\/p>\n When I ask participants whether they have any questions about what they’re seeing, it’s a trick question. I’m not actually going to answer their questions. Rather, I note their questions as content gaps or use it as an opportunity to give\u00a0a follow-up task.<\/p>\n Common participant questions, and my answers:<\/p>\n They ask:<\/strong> Am I doing this right?\u00a0Should I click this?<\/p>\n I answer:<\/strong> Do whatever you normally would do if I wasn’t here watching you.<\/p>\n They ask:<\/strong> What does this do? What is this for?<\/p>\n I answer:<\/strong> What do you<\/em> think it does?<\/p>\n They ask:<\/strong> How does \u2026?<\/p>\n I answer:<\/strong> How do you<\/em> think it should work? What would work best for you<\/em>?<\/p>\n When in doubt about whether to \u201chelp\u201d the participant, answer their question with a question.<\/p>\n If the participant tells me they would call customer service, then I pretend to be customer service. Then have them to talk through their situation or question.<\/p>\n Get comfortable with silence and long pauses. Give participants time to think and finish (or expand) on their thoughts.<\/p>\n If the silence goes on too long (after a minute, maybe), then I will prompt:<\/p>\n Related to long pauses\u2026 Don’t interrupt. Let the participant finish their thought, then count to 2 in your head just to be sure they’re done talking. Often, it’s the next couple sentences that reveal a goldmine of insight.<\/p>\n As facilitator, my goal is to always be as quiet observer as possible so that I don’t interrupt their train of thought or flow.<\/p>\n If the user is clearly stuck, gotten off task, or hasn’t found the right destination after the third attempt, then I will prompt with something like:<\/p>\n Having said all that\u2026 I DO interrupt and redirect the conversation if a participant is rambling, repeating themselves, or has gotten completely off topic.<\/p>\n I never ask which version of a design or content a participant prefers. A usability study is not a concept test, focus group, or an A\/B test\u2014unless you’re conducting the study with literally hundreds of participants. With just 8-10 participants, there\u00a0are simply too few participants for even reliable directional data.<\/p>\n You CAN test the usability of different versions, but, ideally not all versions should be tested by the same participant. But I’m not a fan of that approach either. Rather than testing different versions, I recommend conducting an iterative study instead. That is, test with 3-4 people\u2014identify the problems and solutions, update the prototype\u2014then test again with 3-4 people.\u00a0Repeat this process for as much runway or budget as your project will allow.<\/p>\n I avoid questions that start with the words \u201cWhat do you think…\u201d or \u201cWhat are your thoughts about…\u201d<\/p>\n Those types of questions are valid, just not in a usability study. Thoughts and feelings-type questions are more appropriate for focus groups, one-on-one interviews, or even a follow-up survey or conversation.<\/p>\n The job of the usability study is to identify whether the system, site, or app is doing its job, not how it makes them feel.<\/p>\n I DO think it’s appropriate to ask questions like:<\/p>\n Those types of questions are very different from:<\/p>\n The first set of questions is still trying to uncover whether the correct information or intent is coming through clearly.<\/p>\n The second set of questions is trying to understand whether the participant likes something or not.<\/p>\n The vast majority of people are not<\/em> designers. It’s difficult (and sometimes uncomfortable) for most usability participants to be put on the spot that way.<\/p>\n Whether I\u2019m conducting a customer interview or a job interview, I aim for a 80\/20 listening-to-talking ratio. That is, I listen 80% of the time and talk 20%.<\/p>\n You’re paying a lot of money to have the privilege of your customer\u2019s undivided attention for an hour. Stop talking and listen to what they have to say!<\/p>\n I find it’s usually not even helpful to the conversation to chime in with \u201cme too\u201d statements. Participants don’t care about your experiences. Rather, I make sure they feel heard and valued using active listening cues:<\/p>\n When talking with participants\u00a0in person or via video chat, I make eye contact, point my knees and feet in their direction, and take minimal notes so they\u2019re not looking at the top of my head the whole time.<\/p>\n There is a lot to think about during a usability test session. A well-designed discussion guide is critical to ensuring the highest return on investment and gain maximum learnings.<\/p>\n\n\nExample leading questions:<\/strong><\/h4>\n
\n
\n
\n
2) Don’t create leading tasks.<\/strong><\/h2>\n
Example of a leading task:<\/strong><\/h4>\n
\n
Example of a non-leading task:<\/strong><\/h4>\n
3) Don’t use website keywords.<\/strong><\/h2>\n
\n
4) Answer questions with more questions.<\/strong><\/h2>\n
5) Don’t feel pressured to fill silence.<\/strong><\/h2>\n
\n
6) Don\u2019t pull a Kanye West.\u00a0Let participants finish.<\/strong><\/h2>\n
\n
7) Don’t treat the usability study\u00a0like\u00a0an A\/B test.<\/strong><\/h2>\n
8) Don’t ask for the participant\u2019s opinion.<\/strong><\/h2>\n
\n
\n
9) Don’t ask participants to design alternative solutions.<\/strong><\/h2>\n
10) Don’t talk so much.<\/strong><\/h2>\n
\n
In Conclusion<\/strong><\/h2>\n
Related Articles<\/h2>\n\n\n