Card Sorting: Uncover Users' Mental Models for Better Information Architecture

Part of making a site easy to use is organizing information so that people find what they’re looking for. Too often, content is structured based on what makes sense to the company, not to the users. (This was the #1 usability problem in our recent study of 43 websites.) One of the primary ways to figure out an organization scheme that best matches users’ mental model is through card sorting.

Definition: Card sorting is a UX research method in which study participants group individual labels written on notecards according to criteria that make sense to them.  This method uncovers how the target audience’s domain knowledge is structured, and it serves to create an information architecture that matches users’ expectations.

Let’s imagine that you’re designing a car-rental site. Your company offers around 60 vehicle models that customers can choose from. How would you organize those vehicles into categories that people can browse to quickly find their ideal car rental? Your company might use technical terms such as family car, executive car, and full-size luxury car. But your users might have no idea of the difference between some of those categories. This is where card sorting can help: ask your users to organize vehicles into groups that make sense to them, and, then, see what patterns emerge.

Conducting a Card Sort

Generally, the process works as follows:

1. Choose a set of topics. The set should include 40­–80 items that represent the main content on the site. Write each topic on an individual index card.
   Tip: Avoid topics that contain the same words;  participants will tend to group those cards together.
2. User organizes topics into groups. Shuffle the cards and give them to the participant. Ask the user to look at the cards one at a time and place cards that belong together into piles. Some piles can be big, others small. If the participant isn’t sure about a card, or doesn’t know what it means, it’s ok to leave it off to the side. It’s better to have a set of “unknown” or “unsure” cards than to randomly group cards.
   ​Notes:
   • There is no preset number of piles to aim for. Some users may create many small piles, others may end up with a few big ones. It all depends on their individual mental models.
   • Users should be aware that it’s OK change their mind as they work: they can move a card from one pile to another, merge two piles, split a pile into several new piles, and so on. Card sorting is a bottom–up process, and false starts are to be expected.
3. User names the groups. Once the participant has grouped all the cards to her satisfaction, give her blank cards and ask her to write down a name for each group she created. This step will reveal the user’s mental model of the topic space. You may get a few ideas for navigation categories, but don’t expect participants to create effective labels.
   Tip: It’s important to do this naming step after all the groups have been created, so that the user doesn’t lock herself in to categories while she’s still working; she should be free to rearrange her groups at any moment.  
4. Debrief the user. (This step is optional, but highly recommended.) Ask users to explain the rationale behind the groups they created. Additional questions may include:
   • Were any items especially easy or difficult to place?
   • Did any items seem to belong in two or more groups?
   • What thoughts do you have about the items left unsorted (if any)?
   You can also ask the user to think out loud while they perform the original sorting. Doing so provides detailed information, but also takes time to analyze. For example, you might hear the user say, “I might put card Tomatoes into pile Vegetables. But wait, they are really a fruit,  they don’t really fit there. I think Fruits is a better match.” Such a statement would allow you to conclude that the user did consider Vegetables a decent match for Tomatoes, even though Fruits was even better. This information could push you into crosslinking from Vegetables to Fruits or maybe even assigning the item to Vegetables if there are other reasons leaning in that direction.
5. If needed, ask the user for more-practical group sizes. You should not impose your own wishes or biases upon the participant during the original sorting (steps 1–3), but once the user’s preferred grouping has been defined, and after the initial debrief, you can definitely ask the participant to break up large groups into smaller subgroups. Or the opposite: to group small groups into larger categories.
6. Repeat with 15–20 users. You’ll need enough users to detect patterns in users’ mental models. We recommend 15 participants for card sorting: with more, you’ll get diminishing returns for each additional user; with fewer, you won’t have enough data to reveal overlapping patterns in organization schemes.
7. Analyze the data. Once you have all the data, look for common groups, category names or themes, and for items that were frequently paired together. If you see that some items were frequently left off to the side, determine whether it’s because the card labels weren’t clear or the content seemed unrelated to the rest of the topics. Combine the patterns you see with your qualitative insights from the debrief, and you’ll be in a better position to understand what organization system will be most successful for your users. (There’s a lot more to the analysis of card-sorting results, but that’s a topic for another article.)

Variations in Card Sorting

Variations in card sorting involve whether or not users can create their own category names, whether a facilitator moderates the session, and whether the study is conducted with paper or a digital tool. Each has its own benefits and disadvantages, which we’ll briefly outline below.

Open Card Sorting vs. Closed Card Sorting

• Open card sorting is the most common type of card sort and what we described above. Generally, when practitioners use the term card sort, it’s implied that it will be an open card sort. In an open card sort, users are free to assign whatever names they want to the groups they’ve created with the cards in the stack.
• Closed card sorting is a variation where users are given a predetermined set of category names, and they are asked to organize the individual cards into these predetermined categories. Closed card sorting does not reveal how users conceptualize a set of topics. Instead, it is used to evaluate how well an existing category structure supports the content, from a user’s perspective. A critique of the closed card sort is that it tests users’ ability to fit the content into the “correct” bucket ­— to users, it can feel more like solving a puzzle than like naturally matching content to categories. The method does not reflect how users naturally browse content, which is to first scan categories and make a selection based on information scent. Instead of closed card sorting, we recommend tree testing (also known as reverse card sorting) as a way to evaluate navigation categories.

Moderated vs. Unmoderated Card Sorting

• Moderated card sorting includes step 4 in the process outlined above: the debrief (and/or think-aloud during the actual sorting). This step is a highly valuable opportunity to gain qualitative insights into users’ rationale for their groupings. You can ask questions, probe for further understanding, and ask about specific cards, as needed. If it’s feasible for your schedule and budget, we recommend moderating your card sorts to get these insights.
• Unmoderated card sorting involves users organizing content into groups on their own, usually via an online tool, with no interaction with a facilitator. It is generally faster and less expensive than moderated card sorting, for the simple reason that it doesn’t require a researcher to speak with each user. Unmoderated card sorting can be useful as a supplement to moderated card sorting sessions. For example, imagine a study involved highly distinct audience groups, and the research team decided to run a card sort with 60 users: 20 users for each of 3 different audience groups. In this case, it can be cost-prohibitive to run 60 moderated card-sorting sessions. Instead, the team may decide to do a small study of 5–10 moderated sessions for each audience group, followed by unmoderated card sorting for the remaining sessions.

Paper vs. Digital Card Sorting

• Paper card sorting is the traditional form of card sorting. Topics are written on index cards and users are asked to create their group on a large workspace. The biggest advantage to paper card sorting is that there is no learning curve for the study participants:  all they have to do is stack paper into piles on a table. It’s a forgiving and flexible process: users can easily move cards around or even start over. It’s also easier for people to manipulate a very large number of cards on a big table than it is to manipulate many objects on a computer screen that often can’t show everything within a single view.  The downside of paper card sorting is that the researchers have to manually document each participant’s groups and input them into a tool for analysis.
• Digital card sorting uses software or a web-based tool to simulate topic cards, which users then drag and drop into groups. This method is generally the easiest for researchers, because the software can analyze the results from all the participants and reveal which items were most commonly grouped together, what category names users created, and the likelihood of two items being paired together. The downside is that the usability of the tool can impact the success of the session — technology problems can cause frustration or even prevent users from creating the exact groups that they want.

Card sorting is a well-established method within the information-architecture field. In fact, a card-sorting study done today will look exactly the same as the photo of a study we did 23 years ago — at least if using physical cards and not a software solution. There are many ways to run a study and the variations above are by no means a comprehensive account of all the possible types of card sorts, but they are the most common. When planning a card-sorting study, you’ll want to pick the approach that is best suited to your goals and resources.

Conclusion

Card sorting is a highly useful technique in information architecture; it is used to understand how users think about your content. It can help you organize content so that it suits your users’ mental models, rather than the point of view of your company. Card sorting can be supplemented with other information-architecture methods to identify issues in your category structure.