Usability testing is a popular UX research methodology.

In a usability-testing session, a researcher (called a “facilitator” or a “moderator”) asks a participant to perform tasks, usually using one or more specific user interfaces. While the participant completes each task, the researcher observes the participant’s behavior and listens for feedback.

The phrase “usability testing” is often used interchangeably with “user testing.”

(One objection sometimes raised against the phrase “user testing” is that it sounds like researchers are testing the participant — we never test the user, only the interface. However, the term is intended to mean testing with users, which is exactly the point of empirical studies.)

Why Usability Test?

The goals of usability testing vary by study, but they usually include:

  • Identifying problems in the design of the product or service
  • Uncovering opportunities to improve
  • Learning about the target user’s behavior and preferences
Usability testing helps us to uncover problems, discover opportunities, and learn about users.

Why do we need to do usability testing? Won’t a good professional UX designer know how to design a great user interface? Even the best UX designers can’t design a perfect — or even good enough — user experience without iterative design driven by observations of real users and of their interactions with the design.

There are many variables in designing a modern user interface and there are even more variables in the human brain. The total number of combinations is huge. The only way to get UX design right is to test it.

Elements of Usability Testing

There are many different types of usability testing, but the core elements in most usability tests are the facilitator, the tasks, and the participant.

A usability-testing session involves a participant and a facilitator who gives tasks to the participant and observes the participant’s behavior.

The facilitator administers tasks to the participant. As the participant performs these tasks, the facilitator observes the participant’s behavior and listens for feedback. The facilitator may also ask followup questions to elicit detail from the participant.

In a usability test, the facilitator gives instructions and task scenarios to the participant. The participant provides behavioral and verbal feedback about the interface while he performs those tasks.

Facilitator

The facilitator guides the participant through the test process. She gives instructions, answers the participant’s questions, and asks followup questions.

The facilitator works to ensure that the test results in high-quality, valid data, without accidentally influencing the participant’s behavior. Achieving this balance is difficult and requires training.

(In one form of remote usability testing, called remote unmoderated testing, an application may perform some of the facilitator’s roles.)

Tasks

The tasks in a usability test are realistic activities that the participant might perform in real life. They can be very specific or very open-ended, depending on the research questions and the type of usability testing.

Examples of tasks from real usability studies:

  • Your printer is showing “Error 5200”. How can you get rid of the error message?
  • You're considering opening a new credit card with Wells Fargo. Please visit wellsfargo.com and decide which credit card you might want to open, if any.
  • You’ve been told you need to speak to Tyler Smith from the Project Management department. Use the intranet to find out where they are located. Tell the researcher your answer.

Task wording is very important in usability testing. Small errors in the phrasing of a task can cause the participant to misunderstand what they’re asked to do or can influence how participants perform the task (a psychological phenomenon called priming).

Task instructions can be delivered to the participant verbally (the facilitator might read them) or can be handed to a participant written on task sheets. We often ask participants to read the task instructions out loud. This helps ensure that the participant reads the instructions completely, and helps the researchers with their notetaking, because they always know which task the user is performing.

Participant

The participant should be a realistic user of the product or service being studied. That might mean that the user is already using the product or service in real life. Alternatively, in some cases, the participant might just have a similar background to the target user group, or might have the same needs, even if he isn’t already a user of the product.

Participants are often asked to think out loud during usability testing (called the “think-aloud method”). The facilitator might ask the participants to narrate their actions and thoughts as they perform tasks. The goal of this approach is to understand participants’ behaviors, goals, thoughts, and motivations.

In this usability-test session, the participant sits on the left, and the facilitator sits on the right. The participant uses a special testing laptop, which is running screen-recording software. The laptop has a webcam to capture the participant’s facial expressions and is connected to an external monitor for the facilitator. The facilitator listens to his feedback, administers tasks, and takes notes. The photo captures the moment after the participant’s task, when the facilitator is asking him followup questions.

Types of Usability Testing

Qualitative vs. Quantitative

Usability testing can be either qualitative or quantitative.

Qualitative usability testing focuses on collecting insights, findings, and anecdotes about how people use the product or service. Qualitative usability testing is best for discovering problems in the user experience. This form of usability testing is more common than quantitative usability testing.

Quantitative usability testing focuses on collecting metrics that describe the user experience. Two of the metrics most commonly collected in quantitative usability testing are task success and time on task. Quantitative usability testing is best for collecting benchmarks.

The number of participants needed for a usability test varies depending on the type of study. For a typical qualitative usability study of a single user group, we recommend using five participants to uncover the majority of the most common problems in the product.

Remote vs. In-Person Testing

Remote usability tests are popular because they often require less time and money than in-person studies. There are two types of remote usability testing: moderated and unmoderated.

Remote moderated usability tests work very similarly to in-person studies. The facilitator still interacts with the participant and asks her to perform tasks. However, the facilitator and participant are in different physical locations. Usually, moderated tests can be performed using screen-sharing software like Skype or GoToMeeting.

Remote unmoderated remote usability tests do not have the same facilitator–participant interaction as an in-person or moderated tests. The researcher uses a dedicated online remote-testing tool to set up written tasks for the participant. Then, the participant completes those tasks alone on her own time. The testing tool delivers the task instructions and any followup questions. After the participant completes her test, the researcher receives a recording of the session, along with metrics like task success.

In remote unmoderated usability testing, the flow of information changes because the facilitator does not interact with the participant in the same way as an in a moderated test. The testing platform takes on the role of the facilitator, administering tasks to the participant. The researcher designs the study and upload task instructions on the platform, and then reviews the data after it’s collected, usually by observing video recordings of the tasks.

Cost of Usability Testing

Simple, “discount” usability studies can be inexpensive, though you usually must pay a few hundred dollars as incentives to participants. The testing session can take place in a conference room, and the simplest study will take 3 days of your time (assuming that you have already learned how to do it, and you have access to participants):

  • Day 1: Plan the study
  • Day 2: Test the 5 users
  • Day 3: Analyze the findings and convert them into redesign recommendations for the next iteration

On the other hand, more-expensive research is sometimes required, and the cost can run into several hundred thousand dollars for the most elaborate studies.

Things that add cost include:

The return on investment (ROI) for advanced studies can still be high, though usually not as high as that for simple studies.

NN/g Resources for Usability Testing

Facilitating a Usability Test

For hands-on training and help honing your facilitation skills, check out our full-day course on usability testing.

Recruiting Participants

Remote Usability Testing

For detailed help planning, conducting, and analyzing remote user testing, check out our full-day seminar: Remote Usability Testing.

Special Usability Testing Studies or User Groups

Printable Usability Testing Poster

You can download and print a poster that explains usability testing (available below for your preferred size printer paper: A4 size or US letter size, or you can scale the printout for bigger sheets).