Involving stakeholders and team members in usability testing is essential in order to foster collaboration and buy-in. At all project stages, the design team should be able to participate in a variety of research activities, ranging from setting goals or defining the target audience to selecting the methodology to use for research and observing the test sessions.
While team involvement is vital in usability research, sometimes stakeholders behave in ways that violate good practices of user testing. Even though inappropriate requests or behaviors are often well intentioned, UX researchers have an obligation to handle these situations delicately.
Ethical User Research
The norms of conduct for working with human participants help us distinguish between acceptable and unacceptable behavior at all stages of research. Adhering to the standards help the UX field uphold the quality of the research and protects the organization from potential legal implications. While codes of conduct vary for each profession, the following serve as guides for helping UX researchers make the right decisions when faced with unethical requests:
- Avoid harming people. Do not expose participants to unreasonable physical or mental stress.
- Respect people’s dignity. Do not make derogatory remarks about clients, participants, or colleagues.
- Act with integrity. Conduct research and report findings in a fair and truthful manner.
- Maintain confidentiality. Respect the privacy and anonymity of the participants.
The Dilemmas and Resolving Them
The above bullet points surely sound reasonable. Who would be so nasty as to violate these basic principles for treating research participants ethically? Most of us, that’s whom, unless we stiffen our spines and recognize the slippery slopes that may sound reasonable at first but end up in the wrong place.
Below are examples of difficult situations and how to handle them:
Team insists that participants continue the study even when they don’t want to. This behavior may sound extreme but unfortunately it does happen. Study sponsors are motivated to get the most out of their money, and letting a participant leave before the scheduled end time means less or no data.
What to do: Hopefully, you will never experience this situation. Users should ALWAYS be allowed to take a break or leave for any reason. At Nielsen Norman Group, we explicitly tell people at the beginning at each session that they can stop at any time.
A person who is no longer interested in participating should never be coerced in staying because doing so is unethical (and probably illegal). Plus, their attitude will taint the findings anyway.
Team wants you to lie to users. UX Researchers are sometimes pressured to lie to users, often with good intent. For example, we are asked to tell participants that no one else is watching when observers are in the other room (or over the internet). The assumption is that white lies are harmless and can benefit the participant by making them feel more at ease.
What to do: In general, do not lie to users. Trust is difficult to earn and easy to lose. Better to be transparent about the setup and testing procedures. Don’t try to ignore awkward situations and keep people wondering. Acknowledge that usability testing might feel unnatural.
If you are recording the session, point out where the cameras are. If your lab has a one-way mirror, come straight out and tell users what it is. And if your colleagues are dialing in, just say so. Don’t put yourself in a situation when a colleague forgets to mute the telephone and the participant hears.
(While you shouldn’t lie to users, there are times when withholding the truth is acceptable. For example, if you’re conducting a competitive study and the user asks who’s sponsoring the study. To avoid biasing the findings, it’s okay to tell the user that you can’t reveal the sponsor at this stage of the study, but that you will inform them at the end of the session.)
Team wants to rescind payment when the participant shows up but is a poor fit. Sometimes you discover that a participant doesn’t fit the profile during the session. In some cases, you might conduct the study anyway to salvage what you can. However, if the participant is too different from the recruitment requirements it’s better to cut your loses and allow them to leave gracefully.
What to do: Regardless of the reason for the recruitment mishap, it’s best to pay the participant the entire incentive. Give participants the benefit of the doubt and assume an error occurred during the recruitment process. The participant put in effort and time to show up at the test location and that should be enough to warrant the incentive. Learn from what happened, make necessary changes to the recruitment procedures, and move on.
The participant’s boss wants to observe the session. Internal studies are nice because you can usually find participants easily. However, be aware that employees might feel uncomfortable being observed by their managers or colleagues. New hires in particular might fear being judged by their superiors or might believe their performance in usability studies will affect their performance review.
What to do: When conducting intranet research, make it a policy to NOT allow bosses to attend sessions of their subordinates. Asking employees for permission is sometimes insufficient because they might feel obligated to conform to the request. Also, make sure any notes, research findings, and reports are anonymized.
Team becomes defensive when users don’t understand the interface. For many designers, it’s difficult to sit quietly (and stare blankly) while someone struggles with the interface you’ve created. It’s tempting to reach over, grab the mouse from the person’s hand and show them how use the site correctly. This is why we usually recommend that you don’t test your own design
We’ve witnessed observers interrupt studies to pitch their ideas to participants in the hopes of changing their mind. This is bad practice because it defeats the point of getting user input. Also, when the team becomes defensive, respondents are less likely to give honest feedback, or, worse, they might feel intimidated. Usability testing is the team’s opportunity to receive feedback, not give it.
What to do: Set clear rules for proper “observer” etiquette. Review the rules with the team before you begin, especially if observers sit in the same room as the participant. Remind observers about the purpose of the study and why it’s important to remain silent and keep an open mind. You can also include these rules as you are explaining the session to the participant: “There are some team members observing the study, but they will be silent for the duration, except at the end of the study when I will see if there are any questions.” Including a phrase like this in the introduction serves as a reminder to all observers on how they should behave.
Allow observers to ask questions at the end of each session. Ideally these questions should be communicated to the moderator, who can reformulate them as needed and present them to the participant. Observers are less likely to interrupt if they know there are opportunities to ask questions later.
Team is distracting during test sessions. When conducting usability tests with participants and observers in the same room, it’s sometimes difficult for observers to remain unobtrusive for the entire session. Sometimes it may be hard to keep a neutral face if the participant makes a funny comment, or, in their excitement, members may want to discuss what they just saw immediately. Such reactions or discussions, even in quiet whispers, distract the participant from their tasks, and even worse, make them feel self-conscious.
What to do: You don’t need a fancy usability lab to conduct usability studies. Often, a simple conference room will do. However, if you have observers sharing the same room, make sure everyone is aware of the rules for participating. Remind observers to remain quiet throughout the session and stay for the entire duration of the study. Keep any movement or sounds to an absolute minimum.
Only allow up to 3 observers in the same room. Users will feel more comfortable when they’re not overly outnumbered. Set expectations by letting participants know about other observers BEFORE they arrive. Try to position observers away from the user’s line of sight. We usually ask team members to sit on the opposite end of the boardroom table and observe the screen using an external monitor.
If you must have more observers, consider broadcasting the session to a second conference room where observers could all gather.
Team jokes about users. When testing shows major flaws in the design, a common reaction from observers is to blame the user. Team members sigh in disbelief as a participant does something completely unexpected and wonder if there’s something wrong with the participant. Occasionally, observers might call the participant undesirable names, such as “guinea pig”, “stupid”, “clueless”, and so on. Name calling is not a joking matter.
What to do: Any negative undercurrent of name calling and labeling is unprofessional. Help team members get in the habit of reframing conversations to what is wrong with the interface, not what is wrong with the user. Create a work culture where all participants are seen as partners and treated with respect.
Ask team members to try being a user sometime. Nothing is better at demonstrating how it feels to be in the participant’s shoes.
Conclusion
Upholding research ethics is everyone’s responsibility. Familiarizing everyone on the team with the code of ethics helps you preserve the integrity of the profession and keep you (and the company) from potential litigation. When you protect the participant, you protect yourself and the company. And you get more valid research results.
Learn how to conduct usability studies and manage challenging ethical situations at our conference.
More on code of conduct, check out these websites:
Share this article: