How to run a good usability test
At Flow we do around 300 usability tests every year – an average of about 6 users per week. Here’s some of the top user testing tips we’ve gathered over the years.
Don’t complete their sentences
Even if you think you know what they’re going to say, you really don’t. Keep quiet and wait for the user to finish talking – even when they’re a bit slow in finishing that sentence.
Don’t use leading questions
“Does that make sense? Do you like that? Was that easy?”
Most people will answer ‘yes’ to that kind of question, because most people want to please the moderator. These kinds of leading questions will lead to false results in your research.
Don’t tell them what they’re looking at
“So this is the products page.”
Rather say: “So talk to me about what you’re seeing here.”
Remember, you’re trying to find out what is going on in their heads, not tell them what’s going on in yours.
Don’t forget about your script
You need to keep one eye on your questions all the time, to make sure you’re covering everything you’re trying to find out. While it is good to let the user explore naturally, remain focused on your goals for the test, and make sure you cover everything you planned to do.
Don’t use the word “would”
“So what would you do if your car broke down?” It’s simply not possible to answer a question like that truthfully. What people think they would do, and what they actually do, are usually very different.
Rather ask: “Has your car ever broken down? What did you do then?” That way the person can tell you what actually happened, rather than having to make something up.
Don’t assume you know what they mean
It is always better to probe a little further than to assume you know what the user means.
User: “I don’t like this form.”
You: “Oh, why not?”
User: “It is long and asks for lots of information.”
You: “And why does that make you not like the form?”
User: “Well, because I don’t have an email address and they always want one!”
You: “Aha!” (not what you assumed is it?)
Don’t use words from the interface when asking them to complete a task
If you want to check whether the user can successfully add something to their cart, don’t say “how would you add this to the cart” when there’s an ‘add to cart’ button on the page. All that tests is their ability to read words.
Rather say, “how would you go about buying this”, or “so what would you do if you wanted to get this”?
Ask one question at a time
“How often do you buy from them? What did you buy there the last time? Were you happy with your purchase?” The poor user won’t know which question to answer. Focus on one thing at a time. Ask your question, and then wait for the user to answer it. You’ll get much richer information.
Use silence to your advantage
Most of the time, keeping quiet will get you a lot more information than if you immediately respond to what they’re saying. People naturally start talking when their conversation partner doesn’t say anything. A well-placed “hmm” or “oh?” can also encourage the user to talk more.
Be an apprentice
Take on the role of an apprentice who truly wants to learn everything they can:
“Oh, and how would you do that?”
“That’s really interesting, can you tell me more about how that works?”
“Wow, I didn’t know that! And what happens after that?”
Ask the 5 whys (or to the point just before it starts getting ridiculous)
User: “I only fly with SAA.”
You: “Oh, why’s that?”
User: “Because they offer great baggage insurance.”
You: “And why does that matter?”
User: “Because I always travel with a surfboard and they’re very expensive and very brittle.”
You: “And so why do you need baggage insurance?”
User: “Well, because airlines are known for being a bit careless with bags and it’s not worth flying if it costs me a surfboard.”
Users often feel unsure if they should try something out in the interface, and look to the moderator for reassurance. Saying “Go for it” or “Let’s give that a go” encourages them to try out what they wanted to without telling them what to do.
Reassure the user that they are helping
Users often worry that they’re not making a useful contribution. If they feel like they are being helpful, you are more likely to get good information from them.
“That’s so useful, thank you.”
“That’s very interesting, please tell me more.”
Return questions to the user
User: “What happens if I click on that?”
You: “Why do you think will happen?”
Articulate your questions carefully
Don’t be afraid to stop and think about what you are saying. Good questions lead to good answers.
Great questions to keep in your back pocket
What do you think about that?
How do you feel about that?
I’m not sure. What do you think? (In response to a question.)
What’s going on here?
Can you show me how you would do that?