The Most Hated Online Advertising Techniques

In 2004, we wrote about the most hated advertising techniques based on research conducted by Christian Rohrer and John Boyd. Online advertising has changed significantly since then, giving us plenty of new formats to test and new questions to ask. With that, we decided to run a study to determine which advertising techniques are most disruptive and detrimental for the modern user experience.

Methodology

We conducted a survey with 452 adult respondents from the United States who were not employed in an IT- or marketing-related industry. In this survey, participants were shown 23 wireframes corresponding to different types of advertisements and rated how much they disliked them on a scale of 1 to 7.

Wireframes were presented for both desktop and mobile variants of the same ad, when possible. If there were no common or practical implementations of an advertisement type on mobile, only the desktop variant was shown. We used wireframes instead of screenshots as our stimuli to avoid influencing users with the ad’s visual design, message, or brand. The wireframes focused people’s attention on the ad format.

The advertisements within the wireframes were bright purple for easy identification and contained placeholder text reading, “This is an advertisement.” The wireframes also had accompanying explanatory text to provide more context. The table below shows the types of ads used in the study and the corresponding explanatory text presented to participants.

After the participants rated all 23 wireframes, they were presented with an open-ended question to elaborate more about the types of advertisements that they particularly liked or disliked. These responses were coded according to type or characteristic of the advertisement (i.e. if additional action was required to dismiss the advertisement, or if it could be dismissed right away — a distinction that we cover later in this article). We extracted patterns from these responses, and revealed specific user requirements for online advertising.

Ad Preference Ratings

In the following analysis, we report ad dislike, which is a number from 1 to 7 (1= strong like, 7 = strong dislike) that directly reflects participants’ ratings. The overall average score for all ads was 5.23. This number shows that our respondents aren’t rabid haters of advertising per se. The overall feeling is slight annoyance; but that’s the average. As we’ll see, certain advertising formats irk users much more and do cross into “hatred” territory.

An ANOVA on ad type and device showed that people hate mobile ads more than they hate desktop ads (desktop average of 5.09 vs. mobile average of 5.45; this difference was statistically significant at p <0.0001). The effect of ad type (discussed later) was also statistically significant (p <0.0001).

None of the ad types were particularly liked: the lowest average rating was 3.81 (just barely better than the neutral point of 4) — that is, most people did not indicate positive affect for any ads, but rather ambivalence toward certain advertisement types.

To understand how ads compared with each other, we ran multiple comparison tests (using Tukey contrasts). We found that, for most advertisement types, the mobile and the desktop counterparts did not differ significantly, with the exception of related links and the prevideo advertisement (p< 0.0001). For the desktop prevideo ad, users could skip the ad, whereas that wasn’t possible with the mobile variant because most native mobile video players do not allow users to skip ads or click on video annotations. Thus, this finding makes it difficult to determine whether it was the mobile condition or the lack of ability to skip the ad which made the ad more annoying. On the other hand, related-links ads were significantly (p<0.05) more disliked on mobile than on desktop, and that difference can reliably be attributed to the device on which they were presented.

Most Hated Advertisements on Desktop

The Tukey contrasts indicated that the top four most hated ads on desktop did not differ from each other significantly, but did differ significantly from all the other ad types (with one exception: deceptive links was not different than top persistent banner), so it would be fair to say that the winner for most hated ad type on desktop is a four-way tie between modal ads, autoplay video ads, intracontent ads which shuffle page content as they load, and deceptive links that look like content but are ads. Pro-tip: don’t run these types of ads if you want people to like you.

As for the ads that were hated the least on desktop: the ratings for nonanimated right-rail ads and related links did not significantly differ from each other, but were significantly lower than all the other types of ads. Running such ads should be safe and not damage users’ brand loyalty.

Another notable finding is that there was no difference in annoyance between the two types of persistent banners (top and bottom). Not surprisingly, animated right-rail ads were rated as more disliked than nonanimated ones.

Most Disliked Ads on Mobile

On mobile, the hierarchy of ad dislike is more complicated. The most disliked ads were modal & intracontent ads with content reorganization — these were rated significantly higher than all other ads (with the exception of prevideo without skip and deceptive links).

Related links was the clear mobile-ad winner: it was rated as least disliked and differed significantly from all other ad ratings.

Prevideo ads (with no skip option) and deceptive links were rated significantly worse than sponsored social media and related links, but were no different than any other links. And although the average rating for top persistent banners was higher than for bottom persistent banners, this difference did not reach statistical significance.

Related links were significantly less disliked (p<0.001) than any other mobile ad type. They are the only advertising format we deem to be completely safe to use on mobile devices.

Open-Ended Participant Responses

We received 330 open-ended responses. Any comments that did not have specific information about advertisements or an opinion toward them were considered unqualified responses and were eliminated from our dataset.

Of the 330 open-ended responses, 232 were qualified responses which were coded to indicate the ad attribute that the participant mentioned (e.g., obstructed content, animated ad) and the affect expressed by the comment (e.g., positive, negative). For example, if a participant indicated she “didn’t mind,” or “was okay” with ads “about products I’ve looked at already,” we coded the response affect as positive and the ad attribute as retargeting ads; or, if a participant indicated that he didn’t like ads that “pop up and cover the screen,” we tagged the comment affect as negative and the ad attribute as obstructing content (since we did not know if the participant referred to nonmodal ads partially covering the screen or to modal ads fully covering the content). Naturally, some respondents had more to say about ads than others, and each ad attribute that was mentioned was tagged in the response.

We then looked at all positive and all negative responses, and broke down the totals into attributes that were tagged. In total, we identified 15 different attributes; these overlapped with some of our ad types.

Most of the ad attributes mentioned were associated with a negative affect only, receiving no positive comments. A few exceptions stood out:

• Right-rail and related links received a majority of positive comments.
  • “I like ads that do not obstruct content. I can glance to the side and decide if I want to open but am annoyed when I don't have that choice.”
  • “I am fond of links to the side and at the end of my pages. I can't tell you why, but I like them and am much more likely to click on them and check them out than anywhere else.”
• Banner ads, ads that could be dismissed through a user action, and video ads that could be skipped received some positive comments, although most of the comments were negative in affect. The distinction here is that users positively viewed advertisements which could be skipped or ignored, in which they had control. However, users negatively viewed advertisements which "forced" an interaction, thereby delaying access to primary content. Perceived control seemed to be the influencing factor over affect.
  • "Ads that pop up in the way, force you to close them, or flash for attention just make me hate the product being advertised. "
  • “Video ads before a video you want to watch are okay as long as they are short and give you an option to skip after so many seconds.”
• Retargeting ads had mixed results – while 64% had negative affect, a large proportion commented that these ads were useful and relevant. The others complained that these ads were no longer useful or otherwise that they were “creepy” and, in some cases, ruined surprise gifts.
  • “And I know lots of people think it's creepy to have ads show up with items you've been searching for, but I like it!”
  • “… I always find it unnecessary when I see ads for the exact product I was searching, it would be better if they showed similar products instead.”
  • “I HATE that something you searched for pops up later on other websites. Way to ruin Christmas!”

The ads that generated most positive comments were those which did not look like ads or were related to the user’s primary task. Participants spoke favorably about advertisements which “blended in” with content, like social media and prevideo ads (with skip). Sponsored social media ads tend to be displayed amidst other posts, formatted nearly identically, and prevideo advertisements were expected and tolerated (if they were not autoplay videos, and could be skipped). However, it’s important to note that deceptive links were vehemently disliked and not trusted for their lack of upfront disclosure. It’s a delicate balance to utilize these findings for advertising creatives: design to blend in without being deceptive.

Mobile vs. Desktop Ads

Advertisements tend to be received more negatively on mobile devices than on desktop. One reason may be the relatively larger footprint that ads have on mobile screens: limited real estate exacerbates the existing usability problems found on desktop. Additionally, the context of mobile use tends to be “on-the-go” — that is, users are more likely to be distracted by competing stimuli and the need for efficiency is drastically increased.

With that in mind, what makes an ad annoying on desktop will make it intolerable on mobile.

Conclusion: Advertising in 2004 vs. Today?

Some things don’t change — users’ expectations, in particular. The popups of the early 2000s have reincarnated as modal windows, and are hated just as viscerally today as they were over a decade ago. Automatically playing audio is received just as negatively today. The following ad characteristics remained just as annoying for participants as they were in the early 2000s:

• Pops up
• Slow loading time
• Covers what you are trying to see
• Moves content around
• Occupies most of the page
• Automatically plays sound

While, as a whole, web usability has improved over these past several years, history repeats and designers make the same mistakes over and over again. Designers and marketers continuously need to walk a line between providing a good user experience and increasing advertising revenue. There is no “correct” answer or golden format for designers to use in order to flawlessly reach audiences; there will inevitably always be resistance to change and a desire for convention and predictability. That said, if, over the course of over ten years, users are still lamenting about the same problems, it’s time we start to take them seriously.

Resources

If you'd like to see these wireframes in more detail, or even use them for your own studies, Nielsen Norman Group reserves rights to these images, but grants your permission to use them or even modify them, as long as you credit us with the original research and reference this study. Below, you can find a .zip file which contains the following:

- PNGs of all the wireframes used during this study
- A PPT which contains the base elements used to create the wireframes
- A ReadMe.txt, which contains copyright information about how you can use these wireframes