A couple years ago when shopping for a rug for my living room, I struggled to envision how each might look in my room. How large do I really want it to be? How would the colors and patterns coordinate with my current decor? At the time, I was lucky to come across a local store owner who let me take a rug home to try it out before purchasing it! Seeing that rug in its final space is what sealed the deal and made me feel confident in making a fairly large purchase.
But what about those times when you can’t view a physical item in a store, let alone bring it home to try it before you buy it? How can you still get a good enough sense of that item to feel confident purchasing it online? Augmented reality (AR) tools, which allow users to either “try on” or view an item in their room by superimposing a digital image of an item over the front- or rear-facing camera view, have the potential to be a game-changing shopping feature as they allow users to preview that item in its prospective environment. However, this technology still has a long way to go before it is truly useful, and subpar past experiences detract from people’s perception of its value.
To better understand how people perceive and interact with augmented-reality features while shopping for physical items online, we conducted a remote moderated study with 10 participants. While some desktop sites do have AR tools, they are more prevalent on mobile devices, so this study focused only on AR features for mobile. Additionally, because some of the AR mobile apps tested were supported only on iOS, we limited the study to users with newer iPhone models that supported the AR technology.
Participants were given broad shopping tasks on specific sites and apps that we knew to contain an AR feature, such as finding a pair of sunglasses or a new desk lamp for their home office. When possible, we matched the tasks given to each participant with their specific interests and shopping needs, which we gathered during the screening process. We used the stepped-task method: we started with a broad task to observe whether the AR feature was discoverable and whether users were interested in interacting with it unprompted. Then, if people did not interact with the AR feature, we asked them how they would know whether the item would work well in their space or look good on them. If they still didn’t interact with it, we prompted them with a “What’s that?” to direct them toward the tool.
In this article, we discuss our findings related to when this technology is useful and how users perceive it. Specific usability guidelines for designing AR tools will be discussed in a future article.
Increased Usefulness due to COVID
To address the elephant in the room: Yes, COVID-19 has changed user behaviors, and so using augmented reality to preview an item in its eventual environment is increasingly useful due to COVID-19 restrictions and a general unease around shopping in person and touching shared surfaces.
Several participants commented about this increased reliance on online shopping. They thought that AR tools seemed particularly useful for bridging the gap of not seeing an item in real life before making a purchase decision. This is especially true for products that most people want to touch or try on before purchase — actions that, even when venturing out into a business in person, still aren’t really possible. For example, most beauty stores no longer set out makeup samples to test, nor would someone feel comfortable putting on a lipstick that a lot of people may have touched. (Not to mention, how could you do that with your mask on?)
For example, after one participant tried on makeup using Ulta’s mobile-app AR feature, she commented, “It’s been a long time since I’ve been to any place to get anything … I can see how, especially now — you know, that it’s hard to get out as much with Covid and everything — that this is really a nice feature.” She later added, “I like this. Because then you could feel more comfortable ordering it online too, as opposed to … having … to see the color in person.”
Products that Benefit from AR
Augmented-reality features are most helpful for items where aesthetics or other physical traits, such as size, are primary factors driving the purchase decision. Home décor and furniture are top candidates to get the AR treatment, as people want to ensure that these items would fit in their space and match their overall style. The ability to preview the item in AR takes away some of the uncertainty of shopping online, and several participants commented that they felt more confident that they would not have to go through the hassle of returning their purchase. For example, one participant looking for a desk lamp on the Target mobile app stated, “It’s really helpful because I get an idea of what it does look like in the space that I wanted it to go, versus getting it home and it might not look right.”
For personal wearable items such as eyeglasses, AR tools can be more helpful than images alone. Even though many sites now strive to include a diversity of models in their product images, those people are never going to look exactly like you. As one user stated while looking at the clipped product images for a pair of sunglasses, “It would have been nice to see them on another human being, you know, a model figure or whatever. But … that’s not me. That’s not my husband. So, I would prefer to try them on myself.”
Similarly, virtual makeup tools are extremely useful, as choosing the best shade of a product requires matching it to the individual’s skin tone (and undertones) in addition to personal style. Foundation-matching tools and other AR features that allowed users to see makeup products applied to their face were perceived as very helpful by participants. These tools made them feel comfortable buying the makeup online and enabled them to step outside their comfort zone and try a wider range of products than they might have otherwise. The AR experience was most helpful for beauty products that were opaque rather than sheer, as these were more visible. As one participant explained while looking at lipstick, “I mean, that [the AR tool] would be maybe more helpful for a less natural one, you know. Like it was so close to my natural lip color that it was kind of hard to tell a difference. So maybe if I tried a wild one […] If I were looking for something outside my comfort zone, it [AR] might be a feature that gives me enough of a push to try it.”
Even though not all physical items benefit from an AR capability, the presence of an AR feature is not detrimental (unless it is difficult to use). Large appliances, like refrigerators or washing machines, didn’t appear to benefit much from AR, as these are first and foremost functional products — the features and tech specs are the primary differentiating factors. Yes, size is critical, as they must fit into a defined space in the user’s home, but that is easy to tell from the product’s written description, and would actually be hard to gauge from AR alone, as the technology to calibrate and accurately scale the item to the user’s environment isn’t quite there yet. However, because these are pricey purchases, any extra information was seen as helpful and users did still appreciate seeing how well the finishes matched their space. One participant also mentioned that she would have liked to interact with the refrigerator — for example by opening the doors to see whether it would hit a nearby cabinet.
Accuracy and Realism Determine Helpfulness
How realistic an item appeared and whether it seemed accurately placed in the environment had a huge impact on the overall perception of the tool’s helpfulness. “Cartoony” tools garnered laughs and didn’t persuade users to purchase the items. For example, a participant trying on earrings virtually on the Kendra Scott mobile website laughed, experimented with a few angles of her head, closed the AR tool and stated, “So, that would not make me have a higher chance to buy the item. I don't know what I expected it to look like, but when I looked straight on [to the camera] it didn't even look like they were on my ears…. Then I tilted it to one side to just see one and then I could kind of get it on my ear. And I just feel like, it looked like really cartoonish.”
Thankfully however, this negative perception of the AR feature didn’t extend to the item: users could separate the (lack of) functionality of the tool from their judgement of the product. The participant on the Kendra Scott website continued her comment: “It wouldn't deter me from buying the product though. Actually, … if I still really liked it, I would just look at it on the other person [model product photo] to get the idea of the size and what it would really look like. And I would kind of think like, ‘Oh, that's a nice try.’ … But it wouldn't deter me either. It just wouldn't increase the chance of me buying it.”
People were aware that AR may not render the item exactly like in real life, so as long as the AR rendering was reasonable, they felt the feature was helpful. A participant shopping for wall art on the Etsy mobile app stated, “I think it looks pretty realistic. I know it’s going to look a little different, but it gives you an idea to start with, so I like it.”
Issues with proportion and scale were not as easily overlooked, because gauging the size of an item is a top reason for using an AR tool. Calibration for outward-facing camera views was difficult (a wall is harder to detect accurately than the very distinct contours of a face), and many tools struggled to balance the tradeoff between simple calibration versus accuracy.
AR features that didn’t calibrate at all tended to just throw flat, noninteractive images of the item in the user’s scene. Because these were not to scale, they did not help users get a sense for how large the item would appear in relation to their space. In general, these 2D View in Room features fell flat (pun intended) and users were not impressed.
“I like being able to interact with the product a little more than just look at the flat picture of it, if I’m doing augmented reality. Like, I really like how on Amazon … you can do the 360 view where you can see the back of the product, you can see underneath the product, and you can zoom in. … You feel like you're inspecting the product in person, you can really investigate the product. So, rather than just superimposing a picture, just having it more interactive — that is helpful.”
On the other hand, AR tools that assessed the camera view and aimed to show items to scale were often difficult to calibrate, to the point where users were forced to give up because they couldn’t get it to work. Further, not being able to place the item in a precise spot, to scale, hurt the overall perception of usefulness. One participant who struggled to maneuver a sofa in Interior Define’s AR mobile app explained, “It’s not letting me put it on the floor, it’s not giving me [accurate] proportions … So, it’s really not helping me that much.”
Another participant struggled to place a television on her wall using the Best Buy mobile app’s AR feature because she couldn’t get the tool to calibrate. After she managed to place it partially on top of her couch, she said, “This isn't really super helpful because I need to see it on the wall and not on the couch, but, I mean, it kind of gives you an idea that it's going to be close to the length of the couch. But it should be able to just put it on the wall. I want to see what it's going to actually look like. Not, you know, what it could maybe look like.”
Past Experiences Drive Expectations
Over time, negative experiences with AR features can hurt future expectations for this technology. Before launching an AR tool, several participants commented that they had played around with such features in the past but never used them seriously as a part of their purchase decision because they were difficult to use and yielded lackluster results. Each challenging encounter with AR technology adds to this perception that it is too much work to be worthwhile. Poor experiences can thus prevent people from taking advantage of AR features. In the words of one participant, “I’ve rarely used [AR] because it takes so much to get it to work, like you saw. But I do think they’re helpful when they work.”
Another participant’s advice to designers was to “work out the bugs. I think that’s a really good feature if it works well; otherwise, it can be really frustrating because we did spend several minutes just trying to get it to pop up there, and, when it did pop up, it was kind of underwhelming.”
Some participants also drew on their past experiences with the brand to make judgements about the AR tool. (This is the halo effect at work!) Brands perceived as high-end or luxury were expected to have better-functioning AR tools compared to budget-friendly companies. For example, one participant was disappointed that the Crate and Barrel mobile app’s View in Room feature merely superimposed a 2D image in the camera view. She stated, “When I think Crate and Barrel, I think they are higher-end than Amazon and Target, so they should be able to do the things that those kinds of places are doing. … their new tech should be at least as good.”
Currently, Would Take It or Leave It
As is, AR technology isn’t refined to the point where it is a major source of information for making a purchase decision. At best, people might look at it, but not weigh it heavily as part of their consideration of the item. One participant explained, “It just seems like a way to enhance your shopping experience, as opposed to the sole way you’re going to make your decision. It … gives you supplemental information to make you want to buy it.”
Several participants commented that the product photos were more helpful than the AR feature, as they often gave a more detailed look at the items and offered a wider range of views. For many, these photos were enough:
“I’m used to always just looking at product photos. I probably would just be happy doing it that way … If it [AR] worked, it’d be nice to be able to see it in the area, but it’s not something we’re used to doing. In the past, I’ve either gone to a store and looked at it [the product], or just used my imagination.”
“The picture is worth a thousand words to me, you know. The ‘virtual assistant’ [AR] — I didn’t have that years ago.”
“I mean, I get a better visual sense when I’m viewing the couch on the website, because there’s more detail. … On the website, I get a better sense of the fabric, I get a better sense for the contours. … I would use this AR part later in the process than the website, because at that point then I’m trying to see if the couch I’m looking at would fit in my actual space.”
Excitement for the Future
Current usability issues aside, people were still fairly excited to see that more and more places were offering AR features and showed faith in the potential of this technology. Participants were fully aware that AR is fairly new, which is likely why they were often quite forgiving when things didn’t quite function the way they had hoped.
In the future, participants hoped that AR could be more interactive and allow them not only to view different angles of an item but also to turn on and off a lamp, or open and close furniture drawers or appliance doors. For instance, after viewing a refrigerator in AR on the Home Depot app, one participant said, “I don't know if this is an option, but if … [you could] open the doors, … [to see] if it'll bang into the counter too much, to see just how much space it's taking … it might be nice to be able to see how the doors might open.”
Conclusion
Augmented reality has the potential to be better than reality, as it allows users to preview items faster and more easily than in real life. For instance, it is much quicker to swipe through multiple makeup looks than to repeatedly wash your face to test the next color. Likewise, AR gives consumers the ability to preview furniture in its final location rather seeing it in a store and having to rely on measurements and imagination.
However, this technology still has a long way to go both in terms of usability and usefulness. Making these tools less clunky and showing accurate sizing would greatly improve their helpfulness. Because negative experiences tend to weigh heavily in people’s minds, it’s wise to wait until an AR feature can be fully refined and tested instead of launching a subpar experience too soon.
In the words of one participant, “I think the only way to do it well is to make it feel very realistic. [When they don’t appear realistic,] it's very neutral and probably, when you're spending that much money on having somebody create that, it's not worth it. So, … If I owned a company, I wouldn't use this [AR] or put it on my website until it really helped consumers see what the item was in your space.”
Share this article: