The pros and cons of moderated and unmoderated usability testing – get clued up in less than 10 minutes!
Users shouldn’t have to think too hard when they land on your site or app. By performing usability testing, you can ensure that your interface is effective and efficient, and that your users are satisfied.
Usability testing can be broken down into two basic techniques: moderated and unmoderated usability testing.
For anyone who doesn’t know the difference, this is the post for you. You’ll learn how and when to use these techniques, plus explore the benefits they can bring to your design process. So read on to get informed about the right usability testing technique for your UX project!
Moderated vs unmoderated usability testing: everything you need to know
Part 1: Moderated usability testing
What is moderated usability testing?
Moderated usability testing is a usability testing technique that involves the active participation of a trained facilitator or moderator, and is usually performed in a lab or corporate setting.
Moderated usability testing facilitators need expert knowledge of the product being tested. They are in charge of administering tasks, guiding test participants through each task, recording behaviors and comments they make, answering their questions and replying to their feedback about the test – all in real time.
Moderated usability testing can be done either in-person or remotely. If done remotely, participants will need to share screens with the moderator so that the test can be conducted in real time. Participants are often asked to think aloud as they complete tasks as this helps moderator follow their train of thought and how they would get from touchpoint A to touchpoint B.
Usability testing is an important part of the user experience as it helps UX teams to understand how their target users interact with their product. Moderating usability studies goes one step further in helping UX teams connect with their users and helps them to collect additional metrics about the “how” and “why” of participants’ responses to the study.
This technique is often used earlier on in the design process than unmoderated usability testing. This is because it allows UX teams to ask participants follow up questions about their responses – which is key when working out the overall concept of a design project.
How do you conduct moderated usability testing?
Once you have planned your test and recruited your test participants, it’s time to get to work. You’ll need a laptop, screen recorder and a moderated usability testing tool. Our favorite tools include Validately, Userlytics and Justinmind – get more information about the best tools here.
According to Drupal, studies are usually made up of three parts:
Part One: pre-session questions
Typically, the facilitator or moderator will sit with each test participant, set the ground rules and explain what the goal of the test is.
Then, the facilitator will ask the participant questions based around focus areas that relate to the goal of the test. For example, “What does a participant do with XXX?” and “How does a participant expect the XXX to work?”
Part Two: tasks
Next, test participants dive into the study’s activities laid out by the facilitator. This might involve performing frequent tasks (e.g. creating an account) or critical operations (e.g. removing a user).
As the participant performs these tasks, the moderator will record their behavior. There are several moderating techniques to choose from, as pointed out in this article by Jen Romano Bergstrom.
For instance, Concurrent Think Aloud moderating (CTA). This technique involves participants thinking aloud to keep a running stream of consciousness as they work.
Part Three: follow up questions
Then the facilitator will ask the participant about their overall experience with the feature(s) being tested, using parameters such as effectiveness, efficiency, satisfaction, ease of use etc.
Once the studies are complete, it’s time to analyze and report on test findings in order to make sense of all the data collected. Typically, teams will browse and categorize their notes and prioritize the issues that come up according to their severity. Design iterations may then be performed according to the findings.
Best practices for moderated usability testing
- Facilitators should aim to build a good rapport with the test participant in order to get the best results. As UserTesting points out, the deeper the participant’s trust in the facilitator, the more open they’ll be.
- Plan for test derailment. Problems often arise and being able to reset your test quickly will help you avoid wasted time and flustered participants.
- Use prototypes to test your features. Depending on the stage at which you are performing moderated usability testing, you may want to use a low fidelity wireframe or an interactive prototype (learn the difference here). Either way, keep things visual to keep the user flow natural.
Benefits of moderated usability testing
- Having a moderator in the room with the participant can help to provide clarity as well as make sure that tests get completed even when a task has been derailed
- Observing participants allows for follow up questions and exploring unexpected activity
- This technique also yields a richer understanding of target users as well as an understanding of the feature from a broad range of perspectives
Part 2: Unmoderated usability testing
What is unmoderated usability testing?
Unlike moderated usability testing, unmoderated usability testing is completed by test participants in their own environment without a facilitator present. This technique is typically quicker and cheaper than moderated usability testing (although not always! See our Usability Geek guest post on guerrilla usability testing), making it a good choice for UX projects with tight deadlines and budgets.
As the Nielsen Norman Group explains, there is no real-time support and no opportunity for the participant/facilitator to ask detailed questions unmoderated usability testing. This is why this technique is usually used to test a few specific elements of a product, rather than an overall review.
How do you conduct unmoderated usability testing?
As always, start your usability study by defining its specific objectives, and then designing the questions and tasks.
Note that when performing remote unmoderated usability testing, it’s important that tasks are simple and clearly written. You won’t be able to steer participants in the right direction if anything goes wrong so obtaining useful data relies on you setting clear instructions.
You’ll need to an online tool designed for remote unmoderated usability testing, such as Loop11, UserZoom and MUIQ. These tools will enable you to record user sessions and show the time taken on each touchpoint. Participants will also be able to leave feedback for the UX team to look at once the test is complete.
Learn more about conducting an unmoderated usability study in this video by UX Mastery:
Best practices for unmoderated usability testing
- Consider testing your usability testing tool before sending the test URL out to participants, mocking up a real test situation. You could even try pilot testing on a member of the team who wasn’t involved in designing the test questions.
- Take advantage of the fact that unmoderated usability testing is cheaper faster and by including more test participants. As the Nielsen Norman Group states, no-shows are higher for remote studies than in-person studies.
- Make sure you follow up with test participants post-study. You’ll want to get as much feedback from them as possible, and if you’re not in the room with them, you’ll need to reach out afterward.
Benefits of unmoderated usability testing
- Sessions are shorter than moderated usability testing and results are instant
- You can recruit participants from a wider geographic area for a larger sample of behavior data
- There’s less chance of human error or bias and so results are generally more accurate
Moderated vs unmoderated usability testing: the takeaway
Usability testing is an essential UX practice, helping you uncover barriers in your design process and improve the user experience of your product.
Moderated usability testing gives you the opportunity to interact directly with your target audience and gain a profound understanding of their pain points and goals. The downside of this technique is that it takes longer and is often more expensive, taking into consideration travel and participation costs of test participants.
In contrast, unmoderated usability testing is a quick, cheap and simple approach to collecting data about your users. With immediate turnaround, you can afford to recruit more participants for a wider test sample. But even though you’ll gather more results, your tests will be limited in scope as you’ll need to ensure that every single test instruction is crystal clear.
Whichever usability testing technique you choose, know that you can always conduct your usability studies using your Justinmind prototypes. Justinmind is compatible with all the major usability testing tools available, so you can go from prototype to end product as smoothly as possible. Download now and enjoy all of our Pro features free for 30 days!