Designing for global audiences with different cultural backgrounds can be challenging.

To thrive in an international market, sites must go beyond translation and localization and gain first-hand data on how your users in your markets interact with your products. Even though the main usability guidelines stay the same across countries, testing with international audiences can reveal usability problems specific to those cultures. There are two main reasons to test with international users:

  1. International users may interact with your products differently or rely on specific features more heavily than local audiences.
  2. Mental models and how people interact with technology or organizations can vary from country to country.

Usability Problems Associated with Features More Likely to Be Used by International Users

International usability testing can uncover issues with important site functionalities rarely used by your major group.

For instance, domestic users may rarely if ever switch the site language. But for international users, the findability of this feature is crucial. During our user study in China, a female participant tried to shop at the mobile version of the South Korean clothes retailer Maybe-Baby. Soon, she realized the page was written in English and Korean, none of which she could read. Then she wondered if there was a Chinese version, tapping the navigation menu with no luck. She took a long time to discover the language switcher, which was in the page footer. She kept scrolling on the homepage, and after scrolling for about 13 screenfuls, she finally saw a row of national flags at the bottom of the page. Clicking China’s flag took her to the Chinese version of the site. She said, “I think it’s a good design, showing national flags for different languages, because maybe I don’t know how to say ‘Languages’ in either English or Korean. But why do they put it at the end of such a long page? I may have missed it.”

The mobile version of maybe-baby.co.kr (left) was 13-screenful long, with the language switcher in the footer (zoomed-in in the right part of the screenshot).

Clothing size is another feature that can vary from country to country, as are measurement units of choice. People shopping on a foreign site may feel they need to understand sizing and use tools such as size guides more than domestic users. A study participant tried to shop for a pair of jeans at Zara.cn, but she wasn’t able to properly access it. When she clicked the Size Guide button, an overlay with a big picture of a female wearing a pair of jeans showed up. The user was supposed to choose a size in order to have the dimensions shown on top of the image. But, unfortunately, she did not understand that: the Choose a Size label and the tiny arrow associated with it were very close to the bottom of the page. She was confused, “Where is the guide? It’s just a picture.”

Zara.cn: When a participant clicked Size Guide), she saw an image of the jeans on a model (left). She was supposed to use the Choose a Size below that image to select a size and have the corresponding dimensions overlapped on the image (right). However, she never discovered that functionality because the dropdown was too close to the bottom of her screen and the associated arrow too tiny. She thought there was something wrong with the site.

Uncovering these basic usability issues not only helps with international audiences but also benefits potential domestic users that are not typically recruited in usability tests, like minorities speaking different languages. It actually improves the accessibility of your design.

Culturally Specific Issues

In addition to general usability glitches, usability testing can also identify culturally specific problems during international usability testing related to:

  • Context of use
  • Perception of the product
  • Consumption of content
  • Interaction with your organization

Context of Use

Where do people typically use your products and who is with them? What types of device limitations do they have? You can learn about these questions better when you conduct remote international usability testing where study participants perform tasks at their preferred locations (like their homes or offices) instead of traditional in-lab usability testing.

Contextual factors refer to all artifacts (physical and digital) and people that may influence how users interact with your products or services.

While it’s neither possible nor necessary to figure out how all contextual factors impact the user experience, you may be able to identify issues such as broadband limitations.

For instance, when we conducted remote usability testing with Indian participants recently, we observed that several participants had to wait for a long time to load websites. Fancy websites full of animations, pictures, and videos only made it worse.

When a participant visited the Decathlon site on his mobile phone, he waited 20 seconds for the navigation to load (and it didn’t). The navigation was very complex, with lots of pictures and even an embedded search feature. He finally turned to the site’s search bar instead of waiting for the navigation to load fully. “The website is pretty heavy to load,” he commented.

In contrast, when he tried to shop at Paytmmall.com (an Indian ecommerce platform), the navigation’s minimal design loaded immediately, and he was happy to use it to move around the site.

Decathlon.in used a complex navigation (left). For each main module, there were several interactive submodules with a search feature. The navigation took so long to load that a participant gave up on using it. The lightweight and clear navigation of paytmmall.com (right) loaded much faster and the same participant quickly began shopping.

Your product’s compatibility with users’ preferred devices or software, like operating systems or browsers, is also vital. Since people are so used to these essential tools and have no problems browsing domestic websites, if your product doesn’t work, your users will believe it’s your problem, but not theirs.

We observed several such cases when we conducted ecommerce research study with Chinese participants. For example, a female participant tried to pick a lipstick for her friend’s birthday. She had no issues choosing a lipstick when visiting the mobile version of the Chinese MAC Cosmetics, but couldn’t check out for some reason. The cart button just wouldn’t work. She was disappointed about the site and believed something was wrong with it.

It turned out that she set the Baidu browser as her default browser, and there were some compatibility issues between that browser and the MAC site. According to Baidu Statistics, 9.5% of Chinese mobile traffic goes through the Baidu browser. MAC Cosmetics China may lose this segment of customers without even being aware of it. The researcher prompted her to try with her phone’s built-in browser instead, and the MAC site worked.

(However, you shouldn’t rely only on usability testing to find out compatibility issues; for small user segments, it can be hard to identify them during testing. Teams working on international products should consider these compatibility problems early in the design and development stages by checking platform-usage data.)

Sometimes, usability testing can also reveal how users’ interpersonal interactions and communication when using your products influence their experience.

For instance, during our testing, an Indian senior participant often asked her young daughter for help when she couldn’t find something on websites, since she wasn’t good at English and wasn’t very tech-savvy.

(Typically, we don’t allow other people to interfere with participants when they perform the tasks. However, we make exceptions in cases like this, because it’s a reflection of how the user actually uses digital technology in real life. Another example would be when parents help out young kids with tech issues.)

Perception of the Product

How do your international users perceive the visual design of your products? How do they feel about your products or services before and after performing some tasks? How much do they trust your brand?

People from different cultural backgrounds have different visual-design preferences. Researchers Katharina Reinecke and Krzysztof Gajos collected 2.4 million subjective ratings on website aesthetics from more than 39,000 participants all over the world. They found out that Russian participants significantly preferred websites with lower visual complexity, while people from Mexico and Chile gave higher scores to sites with high visual complexity. The same study also found that monochrome designs are preferred by Finns, Russians, French, and Germans, while Malaysians and Chileans tend to like colorful visuals. Similarly, our previous study in China also found that Chinese participants had a higher tolerance for high visual-complexity websites than foreign users.

Testing your visual designs with your target audiences is a good way to gauge their interests.

Beyond immediate aesthetic appraisal, usability testing can also uncover how users, especially newcomers, perceive the credibility and trustworthiness of your products or services. For example, different target audiences may perceive different design components as trustworthy.

When we tested the Chinese version of the Australian pharmacy website Pharmacy Online with Chinese participants, several people praised its credibility. They appreciated that the site had a navigation UI similar to that of Chinese ecommerce websites and also localized components such as a WeChat official account. These elements made the website look familiar to them.

The Chinese version of pharmacyonline.com.au (top) compared with Taobao’s homepage (bottom): Some participants perceived this localized design as highly credible because it used components familiar to them, such as a WeChat QR-code and Taobao-like information architecture.

Consumption of Content

Can international users easily understand the information presented on your site? Do they like its tone of voice? Do they find the content helpful and how do they make use of it?

Testing your content with your global audiences is especially important if you have multiple, culturally diverse audiences using the same language version of your site (e.g., USA, Australia, UK use the same English version).

For instance, when we tested the Chinese version of the Converse website, a young male participant complained about the labels. Instead of traditional ecommerce labels like Men, Women, and Accessories, Converse China used Dude 男的, Dudette 女的. (It is hard to find English equivalents to these two labels. In Chinese, these words often have a negative connotation, instead of the casual tone indicated by the English words Dude and Dudette.)

The participant, who shopped for fashion shoes a lot and belonged to the target audience of Converse, stated, “It says Dude and Dudette instead of Men and Women. It makes me feel uncomfortable. It’s a little bit casual, and I feel they don’t respect people.”

After translating your content for people speaking different languages, be sure to review its tone of voice as it can influence your branding. Don’t listen only to marketing experts; let several users perform tasks with your design and see how they feel about your tone.

Beyond the content itself and its tone of voice, people from different cultural backgrounds have varied preferences on how the content is organized. For instance, people in high-context cultures like China are used with websites with high information density and may prefer them to those with relatively low information density.

You can also find out whether the content you offer is helpful or whether you need to provide specific, additional content that these target audiences may be used to on similar sites.

For example, an Indian participant wasn’t satisfied with the dimension information (in centimeters) provided for photo frames on Myntra. “This is funny because when we measure the wall décor, we use inches instead of centimeters. So, the size information they provided was not useful.” (Some cultures in India prefer inches, others centimeters.)

The model size provided by Myntra (an Indian ecommerce platform) was in centimeters instead of inches, which wasn’t useful to a participant at all.

During our ecommerce study in China, a participant shopping for Nike shoes wanted a popularity-filter feature, which was available on almost all domestic retailer sites. He complained, “There are no filters on popularity! I want to know which ones sell best, so that I’m wearing the most popular style.”

Interaction with Your Organization

How do your international users typically interact with your organization or with other similar organizations? Are there any channels that tend to be used more frequently than others?

For instance, Indian participants prefer one-time passwords, but Chinese participants login via third-party services like WeChat and AliPay on mobile or by scanning a scan QR code on desktop.

When we ran an in-person university website study in China, a participant tried to figure out York University’s transfer student requirements. She wasn't happy with the website's vague information and was upset to learn that she needed to communicate with the institution through emails: “I don’t think it’s a proper way to communicate with Chinese students. I’d prefer some instant communication methods like an online Open Day through Skype.”

People from different cultures can also prefer different platforms to keep updated with their favorite brands. Open-ended and exploratory usability-testing tasks such as Check out whether any of your favorite brands launched something new can reveal their preferences and browsing patterns.

For instance, during our social media usability testing in the US, we combined such questions with a cognitive-mapping activity. We found that participants believed that organizations were more active on Twitter than on Facebook. Several used Instagram to follow different fashion-related businesses. They didn’t link use Snapchat to engage with businesses at all. In China, WeChat and Weibo (the equivalent of Twitter) are the most used social-media channels for engaging with businesses.

When to Run International Usability Testing

You should conduct usability testing with global audiences early in the product-development stage, when planning your strategy and product launch in a new market. Even paper prototyping can give you rich insights on both culturally specific and general usability issues.

You don’t have to identify all issues in one round of tests. Sometimes, running several rounds of small tests with a different focus is cheaper and easier. For instance, you can test basic usability issues, content, and tone of voice for the first round, while also observing context of use. A second round can focus on testing and prioritizing localized features and components.

After launching the product, you can do testing to research the usability of global components, used for all markets, and further investigate what other features you should localize to keep competitive in the new market.

Once your product has been known to your new market for a while, running quantitative usability testing, often used for benchmarking, can also benefit you. Keep track of your chosen success metrics and monitor them for major changes to evaluate your design work and compare your performance with industry standards.

You can (and should) combine usability testing of your products with other methods such as competitive usability testing, interviews, field studies, diary studies, and other qualitative research methods, to fully understand user needs, mental models, and expectations.

Conclusion

Conducting research with your global audiences can help you uncover issues you can’t find just by testing with domestic users. Perform these studies early and often to ensure success in your international markets.

Reference

Reinecke, K., & Gajos, K. Z. (2014, April). Quantifying visual preferences around the world. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 11-20).