Emojis in Email Subject Lines: Advantage or Impediment? 👍 👎

Emojis are small visual representations of an object or a concept. They first started popping up in text messages, but it wasn’t long until they evolved into a set of mainstream and highly recognizable pictographs used across almost every digital channel, including email.

Many newsletters and marketing emails from companies include emojis in their subject lines. Various email-marketing groups suggest they be used to draw attention to an organization’s email within a crowded inbox, but there is little research about the effectiveness of emojis in capturing attention or how their usage influences customer’s perceptions.

To explore these questions, we conducted two studies:

1. In the first one we explored users’ attitudes towards email subject lines that contained emojis compared with corresponding no-emoji analogs.
2. In the second study, we looked at whether emails with emoji subject lines were more likely to make users want to open them than those with no-emoji subject lines.

Users’ Attitudes to Emojis in Emails

Methodology

Stimuli. We selected 13 real marketing emails (from online retailers) that had emoji subject lines. For each email subject line, we created a corresponding, meaningful no-emoji version by either removing the emoji (for 12 subject lines) or replacing it with an equivalent word (for 1 subject line that would have been meaningless without the emoji).

Procedure. Each email was presented separately as it would appear in an email inbox with the sender name, the subject line, and a portion of the preheader text showing. Other emails around the stimulus being tested were blurred out to focus the participant on the subject line in question.

Participants had to perform two tasks:

1. Describe the email by selecting 3 words from a list of reaction words (a modified Microsoft Desirability Toolkit test containing 18 positive words and 12 negative words).
2. Rate the value and trustworthiness of the email on a scale from 1 to 7 (1=low, 7 =high). The second task was performed only on 4 stimuli (in the interest of keeping the study session short).

The email stimuli were presented in random order.

Participants. We had two groups of participants: an emoji group, with 53 respondents who saw only emails with an emoji, and a no-emoji group, with 54 respondents who saw only the no-emoji versions of our stimuli. All respondents used a laptop or a desktop device to complete the study.

Participants had to choose 3 words from this list to describe each email subject line. (Of course, respondents saw the words in a single mixed list, but, for clarity we’re showing you the words divided into two lists.)

Negative Sentiment Increased for Emoji Emails

To understand whether emails with emojis are perceived differently than the ones without emojis, we looked at the negative sentiment elicited by an email, defined as the difference between the average number of negative words and the average number of positive words that participants associated with that email. (Because the emails in this sample were all marketing emails from ecommerce companies, the general sentiment was predominately negative. Users often have negative emotions toward this type of email, considering it to be spam. However, this does not matter for our findings.)

In general, people selected more positive reaction words and fewer negative reaction words for the emails without emojis compared to those that had emojis. On average, an email with emojis received 2.01 negative words and .99 positive words, whereas an email without emojis received 1.97 negative words and 1.16 positive words. Both of these differences were statistically significant at p < 0.005. Adding an emoji to an email subject line increases the negative sentiment towards that email by 26%. (That’s because the negative sentiment for an email without emojis is 0.81, while the negative sentiment for an email with emojis is 1.02.)

In total, adding emojis to email subject lines increases negative sentiment by 0.21. This number looks small, but remember that our study participants were only asked to assign 3 sentiments per email. So, a fifth of a word is in fact a moderately sized difference. Not enough to condemn subject-line emojis as the worst design idea ever, but enough to make you think at least twice before using one.

The words with the largest differences in the number of selections across stimuli are outlined below. These numbers indicate the average number of times these words were selected by respondents for the 13 emails across both variations. 

Two negative words (Boring and Dull) were selected much more often when participants viewed emails that contained emojis. Two negative words (Frustrating and Confusing) were selected much more often when participants viewed emails that did not contain emojis. A single positive word (Straightforward) was selected much more often when participants viewed the variation of the email that did not contain emojis. Differences between the rest of the words in the word list were less pronounced.

Emails Without Emojis Were Seen as More Valuable

Across the sample of 4 emails where value and trustworthiness were assessed, the emails without emojis tended to be rated as more valuable than emails with emojis (2.45 vs 2.64 on a scale of 1 to 7). This difference was statistically significant when emails were treated as the random factor (“across emails”), but not significant when the participants were treated as the random factor (“across subjects”).

In other words, there is some evidence that emails without emojis are perceived as more valuable. There was no statistically significant difference in how participants rated the trustworthiness of emails with or without emojis. Emails with subject lines containing emojis received an average trust rating of 3.51 while the emails without emojis received an average trust value of 3.43.

Likelihood to Consider Emoji Emails

In addition to our research on users’ reactions to individual emails, we were also interested in knowing if emojis influenced whether users would select a particular email from an inbox.

Methodology

Stimuli. We created two different inboxes containing 16 subject lines from ecommerce marketing emails. Each inbox contained emails from 4 ecommerce stores of a similar type (in one inbox, there were only big-box retailers and in the other, only home-décor retailers). 4 emails were shown for each of the 4 retailers.

Each inbox had 2 versions:

• Emoji version: 8 of the subject lines in the inbox contained emojis (2 from each retailer) and 8 did not. The emoji emails were spread throughout the inbox.
• No-emoji version: This version contained the same subject lines as the emoji version, but the emojis were removed. (We chose emails with subject lines that were still meaningful without the emojis present.)

Procedure. Participants were shown 2 inboxes (big box and décor). For each inbox, they were asked to select two emails that captured their attention and that they would want to open. They were also asked to provide a reason by indicating why they selected each email.

Participants. We had two groups of participants: an emoji group, with 55 respondents who saw only the emoji version of the inboxes, and a no-emoji group, with 58 respondents who saw only the no-emoji versions of our stimuli. All respondents used a laptop or a desktop device to complete the study.

Brand familiarity. To control for the influence of brand familiarity on email selections, we first conducted a brand-awareness survey with an even larger set of big-box and home-décor retailers. In this survey we asked 76 respondents to rate their familiarity with each retailer on a 5-point semantic differential scale (1 = not at all familiar, 5 = extremely familiar). The 4 retailers selected for each retailer type were similar in brand familiarity (with an average awareness rating ranging from 3.77 to 3.96 for big box retailers and 1.58 to 1.84 for home décor retailers). 

Adding Emojis to a Subject Line Does Not Increase Its Probability of Being Selected

One of the goals of the study was to understand whether adding an emoji to a subject line makes it more likely to be opened. To investigate this question, we looked at how often people indicated that they would open the same email in the emoji and no-emoji version of the inbox.  We found that adding an emoji to an email did not increase the average percentage of times that email was selected as a candidate for opening. (On average, each emoji email was selected .04% of the time, whereas, for the corresponding no-emoji email, that number was .03%; p >0.1). In other words, all things being equal (type of store, brand familiarity, position in the inbox, etc.), adding an emoji to the email does not make people more likely to want to open that email.

A Higher Proportion of Emails Selected Included Emojis

We also wanted to understand whether, in a given inbox containing an assortment of emoji and no-emoji emails, the emoji emails get more attention. So, for the emoji versions of our inboxes, we looked at the proportion of participants’ selections: how many of their choices contained emojis and how many did not?

First, we should note that even though we had asked participants to indicate the two emails they would open, some participants made no selections and some selected only 1 email. In fact, the most popular choice was to not select any email at all (on average, 58% of the choice opportunities were no selection at all). This high number is probably an artifact of the study, because all emails shown were marketing emails, which many people consider spam. However, this proportion (58%) could even be artificially low for this scenario because participants were directed to select two emails, so it’s possible that more people selected emails than would have in a real situation.

The rest of the choice opportunities were heavily skewed in the direction of the emoji emails: 33% of choices were emoji subject lines, and 9% of choices were no-emoji ones. This difference was statistically significant by a paired t-test that compared the average percentage of emoji vs. no-emoji selections per participant (p <0.00001). In other words, this finding suggests that, when users do choose to consider an email in a mixed, balanced inbox, they are more likely to direct attention to an emoji email.

(Note, however, that we tested only two inboxes; these inboxes had an equal number of emoji and no-emoji emails and we did not control for the position or type of emoji subject lines in the inbox, so it’s unclear if this finding will generalize to other inboxes.)

Emojis Increase People’s Focus on Visuals

Study participants were also asked to indicate why they selected each email. We did a thematic analysis of their free-form responses and identified two main categories which captured more than 75% of the responses:

• Visual: People said their selection was made based on some visual characteristics of the selected email (e.g., emoji, color, capitalization, bolding)
• Meaning: People indicated that the meaning of the subject line (e.g., big promotion).
• Other: A small segment of the responses included other factors for selecting an email, such as familiarity with the brand or placement of the email.

We found that participants who saw the emoji inboxes relied more on visual aspects for selecting an email, whereas participants in the no-emoji condition reported using meaning more to select an email.

This general finding was corroborated by the in-depth per-email analysis. For each email, we looked at how many more visual comments it received compared with meaning comments. (Specifically, to calibrate for the different number of participants in the two groups, we looked at the difference between (1) the percentage of visual comments received by an email out of all possible comments, including empty ones; and (2) the percentage of meaning comments received by an email out of all possible comments, including the empty ones.)  We found that, on average, this difference was 2.8 times bigger in the emoji condition than in the no-emoji condition — meaning that people reported relying more on visuals than on meaning when the email included an emoji. (The average difference was -1.66% in the no-emoji condition, with the negative number indicating that people relied more on meaning, and 3% in the emoji condition; this difference was statistically significant p <0.05. The small percentages reflect the fact that most people left empty comments.)

Thus, adding an emoji to a subject line significantly increased the chance that participants would attribute their selection to the visual characteristics of the subject line rather than the meaning of the email.

Discussion

Before we dive into a discussion of what these results might mean, let’s summarize our findings:

1. Emoji emails were perceived more negatively than no-emoji emails.
2. Emoji emails were seen as less valuable than no-emoji ones, but there was no difference in participants’ perception of trustworthiness for these 2 email types.
3. Adding an emoji to the same subject line did not increase the percentage of times that email was selected.
4. Emojis attracted attention in a balanced inbox containing both emoji and no-emoji subject lines. On average, emails with emoji subject lines were considered for opening more often than different emails without emoji subject lines.
5. Emojis made people more likely to say they would open that email for its visual qualities rather than for its meaning ones.

We should acknowledge the limitations of these studies:

• We only used (unwanted) ecommerce marketing emails. Even though emojis are used in emails and newsletters from other types of websites, we did not have a large enough sample of similar emails outside of ecommerce. Thus, to ensure that all emails tested were of the same type, we restricted the study to ecommerce marketing emails, where emojis are widely used. Moreover, these were not necessarily emails newsletters that our participants had subscribed to, so the impact of emojis may be different if the emails are actually perceived as valuable or wanted. (That being said, many people are unknowingly subscribed to marketing newsletters when they first use a site or, sometimes stay subscribed to these newsletters even though the content is not perceived as relevant, so our study situation may not be that unrealistic.)
• We did not control for the type and number of emojis used. Even though we did our best to include a variety of emojis, it’s possible that these findings do not generalize to other types of emojis (for example, of different colors than the ones used in the study). Moreover, some emails had one emoji, others had several.  And for most but one of the emails in our study, the emojis were redundant — they could be removed with no loss of meaning. It’s possible that our findings would be different if emojis contributed in an essential way to the meaning of the subject line.
• Only 4 emails were included in the analysis of value and trust. To limit the length of the survey, participants were only asked to rate the value and trust on 4 of the 14 emails. These 4 emails may not be representative of all types of emojis users would see in ecommerce marketing emails. However, even with this small sample of emails, the higher averages of value static emails without emojis is a compelling finding.
• We used only two balanced inboxes (with an equal number of emails with and without emojis) with fixed placement of emoji emails. This constraint is likely to affect finding #4 above.
• We asked people which email they would open instead of actually having them open the email. It’s possible that their picks would actually be different in a real situation, when there would be no pressure to verbalize their choice and they may be able to act more impulsively.
• Some of the comparisons in this study are between subjects. Findings 1 and 2 are based on comparisons between people who saw only emoji stimuli and people who saw only no-emoji stimuli. It could be that emoji text shines only in comparison with no-emoji text— for example, it could be that if someone sees an email with an emoji next to an email without an emoji, the emoji email will seem more friendly, but if she sees only two emoji emails, none will seem friendly.

Findings 3 and 5 in the list above are somewhat contradictory. On the one hand side, simply adding an emoji to a subject line does not necessarily make more people say they want to open that particular email. On the other hand, in an inbox that contains an equal number of emoji and no-emoji emails, emojis do seem to attract more of the users’ attention due to their visual qualities.

Moreover, our results suggest that people’s overall attitudes toward emojis are negative and they are perceived as less valuable. There could be several reasons for the negative sentiment toward emojis. In many instances, it’s clear that emojis are being used entirely for their bright colors and ability to grab attention. Even though their attention may be driven to these emails, users don’t like to feel manipulated. It’s possible that the blatant use of meaningless imagery for the sole purpose of making the email stand out above others is perceived by users as a gimmick or a cheap attempt to manipulate their behavior and get attention. It’s also possible that emojis are associated with texting and informal communication, and so they lessen the impression of professionalism of the marketing email.

In a practical sense, even though emojis may attract attention, it’s not advisable to jeopardize users’ trust and add them gratuitously to subject lines.  

Our second study also found that, when both emoji and no-emoji subject lines are present, people say they rely more on visual than on meaning aspects of an email when they choose which email they would be interested in opening. There are two possible interpretations for this finding:

1. They may not necessarily ignore the meaning of the email, but rather they may just have something more salient to comment on when there’s an emoji.
2. People may truly ignore the meaning of an email subject a little more when there’s an emoji involved.  So, to some extent, the emoji may detract from the meaning of the message.

There are several other pieces of reaction-word data that concur with the second interpretation:

• High usage of the words Dull and Boring to describe emoji emails (compared with no-emoji emails). The words Dull and Boring were selected more frequently to describe emails that contained emojis. This seems counterintuitive at first. However, if you consider that the respondents who selected these words were only shown emails with emojis, this finding becomes more interesting:  it’s possible that respondents who were shown emails with emojis tended to be more focused on the visual aspects of the subject lines rather than the actual meaning of the subject lines. Because of that. those subject lines that used more basic emojis were seen as boring or dull compared to their background, pre-study experience of complex, detailed, or clever emojis. We don’t believe this finding indicates that emojis make emails dull and boring across the board.

• Higher usage of the words Frustrating, Confusing, and Straightforward for emails without emojis (compared with emoji emails). The meaning-oriented words Frustrating, Confusing, and Straightforward were selected much more frequently to describe emails without emojis. This finding could indicate that, when viewing emails without emojis, users were more focused on the actual meaning and wording used in the subject lines. Moreover, because of this focus on the meaning of the subject line, participants might have found the emails in our study frustrating, simply because many consider ecommerce sale emails to be spam. This annoyance factor may have been less of an issue with emojis that detracted from the meaning of the email.

Thus, it’s possible that a lack of focus on meaning could have good and bad effects — decreased scannability and an expectation for clever, high-quality visuals, but also, due to more focus placed on visuals(bad), and a dilution of the frustration commonly associated with marketing emails (good).

Using an emoji to support the meaning of the subject line might be an instance where there is value in its usage. More studies are needed in order to understand how emojis affect text comprehension and whether we can speed it up or bolster it with emojis.

Guidelines for Emoji Use in Subject Lines

Based on these findings, we can suggest several guidelines for the use of emojis in subject lines.

• Do not add emojis in email subject lines purely for the purpose of jazzing them up to draw users’ attention. Even though an emoji may attract attention, it may actually make users see your email as less valuable and look more negatively on it, and by association, on your organization.
• Only use emojis in instances where they will truly add value by providing context or relevant emotional value.
• Even if your use of emojis is legitimate, be selective about how often you send out emoji emails. We’ve seen that emoji subject lines may attract attention compared to no-emojis due to their visual qualities, but if every email in the inbox will have an emoji, there won’t be any benefit. So use them sparingly so that you can benefit from them occasionally. Plus, it’s best not to continually push out content that could be perceived negatively.

Overall, emojis seem to be a double-edged sword: they can negatively impact users’ attitudes, but they can also attract some attention and move the focus on the visual aspects of the email (which, in itself could be good or bad). Overusage will wipe out their benefits, likely leaving us only with the drawbacks.