Bush vs. Kerry: Email Newsletters Rated

George W. Bush and John Kerry are both underfunding one of their best opportunities to get out the vote and get elected. The campaigns have millions of subscribers to their email newsletters and yet don't treat them as a proper strategic resource.

How many people change their vote on the basis of a TV commercial? Probably fewer than change their vote (or decide to vote to begin with) because of a trusted friend's convincing argument. Email newsletters offer a direct line to such every-day influencers around the country.

A political campaign's website has three main goals:

• Energize the party faithful. Get them excited about the candidate and give them talking points to influence their friends and family.
• Collect donations.
• Answer undecided voters' questions, and hopefully convince them to support the candidate.

The third goal is mostly important for primaries and less-important races. For presidential elections, fewer people are both undecided and sufficiently interested in politics to actively seek out websites that contain information on the candidates' positions.

Keeping supporters fired up and supplied with arguments is a perfect job for email newsletters for two reasons: they can form an emotional bond with subscribers, and they can immediately react to the news. It's not surprising that both George W. Bush and John Kerry are publishing email newsletters from their websites. What is surprising is how many mistakes both candidates make in their newsletter designs.

Both campaigns are stuck in last century's understanding of media, and it's obvious in their budget allocations: their email newsletters clearly lack sufficient resources for design, implementation, editorial, and user testing.

Scoring Compliance With Usability Guidelines

To assess the usability of the two newsletters, I scored them for each of the 127 guidelines for email newsletter design and for their subscribe/unsubscribe interfaces. I evaluated the subscribe and unsubscribe interfaces on September 13, 2004, and evaluated the newsletters themselves during a seven-week period from August 2 to September 19, 2004.

The two campaigns' scores are remarkably even, though Bush wins by a nose, scoring 1% higher than Kerry across the full set of usability guidelines. Here are the newsletters' average compliance ratings in the four guideline categories:

These are fairly low scores, considering the importance of newsletters for political campaigns and the huge amounts of money available to presidential campaigns. Each campaign has more than $300 million available to spend over a few months. There's really no excuse for not doing better on newsletter design.

Even though the overall ratings are very close, there are interesting differences in the details: each campaign does well in some areas that are problematic for its opponent.

Subscription Interface

In the interface for new subscribers, Bush gets very low scores. Bush's newsletter registration process is complex and requires users to step through multiple screens. The process further requires that users disclose substantial amounts of personal information, without offering a prominent statement of the site's privacy policy.

Bush also asks users for their personal interests, which can be positive if it results in customized newsletter content. However, the site violates the guideline to be clear about how personal interest information will be used. And, in fact, it seems that the users' selections are almost never used. I registered as a small business owner with an interest in high tech, but not until week 7 did I receive any information specifically related to these two topics. (Several messages were customized with local news, however, based on my subscriber address.)

Still, Bush wins the overall subscription interface comparison because Kerry commits several elementary mistakes. Most strikingly, Kerry doesn't send out a welcome message as soon as the user has registered. Kerry also has an annoying splash page that asks users to disclose their email addresses before offering even a glimpse at the content they might receive. If we've learned anything from our user research on email usability, it's that people are getting very reluctant to give out their email addresses, and sites must be clear about what they offer in return. Premature requests for personal information are extremely pushy.

Newsletter Frequency

In our user research, people's biggest complaints relate to newsletters being published too frequently. Both campaigns risk user dissatisfaction on this count because they bombard users with newsletters. The following chart indicates the number of mailings received during each week of my study:

The Republican National Convention (RNC) took place during week 5 of the study. At that time, it was appropriate for the Bush campaign to send out additional newsletters to keep subscribers informed about convention events. It was also appropriate for the Kerry campaign to send out additional newsletters to its supporters, to rebut the attacks on Kerry during the convention and provide its own spin on events.

Week 5 aside, however, both campaigns send out too many mailings for the average subscriber. People who are extremely interested in politics certainly wouldn't mind near-daily emails, but many others would get overwhelmed. Neither newsletter follows the guideline to offer users a choice of mailing frequency, with a less-daunting newsletter for people who are fans, but not fanatics.

Finally, I subscribed to Kerry's newsletter on August 2 but didn't receive any email until August 16. There was no explanation for this. Perhaps the organization had a technical malfunction, or simply didn't publish newsletters for two full weeks. Either way, it's bad.

Content Analysis

The two newsletters differ dramatically in the content they emphasize. Here is a breakdown of the word count sent to subscribers during the study period, divided into four main content categories:

In my content analysis, positive campaigning promotes a party's own candidate and proposals, whereas negative campaigning attacks the opposing candidate or counterattacks to rebut the opposing candidate's attacks. The basic distinction is between explaining what you'll do if elected and engaging in political infighting.

I was rather generous in scoring content as positive campaigning. For example, I included general feel-good statements, such as Arnold Schwarzenegger relating his pride in becoming a U.S. citizen and John Kerry reporting his excitement in meeting large crowds of supporters during his bus tour.

There were some borderline cases, such as Bush's statement that "I have met too many good doctors, especially OB-GYNS, who are being forced out of practice because of the high cost of lawsuits. To make health care more affordable and accessible, we must pass medical liability reform now." I scored this as positive campaigning because it emphasizes one of Bush's policy proposals, though one can argue that the words "especially OB-GYNS" constitute a veiled dig at John Edwards' previous job. On balance, I judged these words as too veiled to truly constitute negative campaigning. Also, I aimed my content analysis at a higher level, focusing on the main thrust of each paragraph or group of sentences, rather than individual words.

Although the emphasis on positive and negative campaigning is opposite for the two newsletters, it's likely that both parties have chosen the correct emphasis for their respective subscribers. Republicans probably get more excited about the prospect of four more years of Bush policies than about dismissing Kerry. And Democrats probably get more fired up over trying to oust Bush than over Kerry's policy proposals.

Announcements and instructions contain information about using the campaign's website, especially news about new features and upcoming chats. The Bush campaign is particularly active in scheduling chats with various campaign leaders, which can be a good way of generating a sense of community around a website. (I didn't evaluate the actual chat content or the chat interface, both of which are often poorly done on other websites.)

Finally, both newsletters expend significant word count exhorting readers to volunteer and donate money. This is understandable, since this is the payoff to the campaigns, but repeated requests can be overwhelming. Kerry in particular risked both donor fatigue and unsubscribes by sending out two fundraising emails in a single day (August 31).

Attracting Inbox Attention

Both newsletters score very poorly in the battle for inbox attention. Their highly variable publication frequency is a problem in itself, because users can't form expectations about when a newsletter will arrive.

The newsletters' sender and subject lines often horribly violated usability guidelines. Certainly, when you receive email from famous people like John Kerry or Arnold Schwarzenegger, you tend to open it. But what about Blaise Hazelwood, Ed Gillespie, Ken Mehlman, Todd Cranney, or Terry Nelson? These are just a few of the "from" names on the Bush newsletter. I'm sorry to say that even though Ed Gillespie chairs the Republican National Committee, I didn't know who he was. And in any case, the fact that the Bush newsletter uses so many different names degrades a user's ability to recognize it. Kerry fares better, with most newsletters sent by Mary Beth Cahill, though hers is also not a name that stands out in a crowded inbox. The guideline here remains: use an institutional "from" line unless a newsletter is written by a famous personality.

Subject lines were universally lame, with Kerry having the most user-repellant subjects, like "Tonight," "Don't stop now," and "Deadline almost here." Why would anybody think that those messages were anything but spam? Bush had somewhat better subject lines, like "Kerry's Flip Flop Olympics," and "Participate in W ROCKS in Alameda County," though he also had content-free subjects like "Brace Yourselves."

Both Can Do Better

Reportedly, the Bush campaign has more than a million subscribers and Kerry's has more than two million subscribers. Clearly, the campaigns are doing something right. Despite my criticisms, both newsletters receive good scores for content, which is the most important parameter for users: if something is interesting, you can forgive many usability sins.

Still, both sites could substantially increase their newsletter's impact by redesigning their subscription interfaces and paying more attention to their inbox appearance. Millions of less-zealous supporters are not being served with the current design. Whenever your compliance with usability guidelines is below 50%, you know you're in trouble.

In 1996, I wrote a review for The New York Times on the campaign websites for Bill Clinton and Bob Dole. I concluded that the two sites scored about even in usability, and I provided several recommendations for improving each site. Two weeks after the article ran, Clinton's site had been updated to incorporate all of my recommendations. In contrast, Dole's site stayed the same throughout the campaign. We all know who won the 1996 election, so maybe this example will motivate the campaigns to pay closer attention to usability this time around.

Follow-Up

• Analysis of the newsletters during the last week before Election Day
• Comparison of the candidates' newsletters with the usability of a professionally published email newsletter from The Washington Post.
• 6 years later: British election newsletters rated. (They scored higher.)