Do Government Agencies and Non-Profits Get ROI From Usability?

For commercial projects, the case for usability's return on investment (ROI) is clear:

• On average, e-commerce sites double their sales by following e-commerce user experience guidelines.
• Even sites that don't sell online can double the conversion rates for business goals such as getting leads or enticing subscribers to read their email newsletters.
• Particularly bad sites (many B-to-B sites among them) can record even greater improvements in key performance indicators, such as those related to visitor counts, whitepaper downloads, webinar signups, or other stages of the lead pipeline.
• Improving the usability of a company's intranet can save millions of dollars through increased employee productivity.

But what about non-commercial design projects? Are there arguments for having government agencies and non-profit organizations spend money on usability even when they don't earn it back in the traditional sense?

Non-commercial Websites

It is possible to make hard-nosed economic arguments for some parts of non-commercial websites. Anytime a site collects money — even if it's for opera tickets, research reports, or continuing education seminars — it can get more money by increasing conversion rates. Higher conversion rates are a direct result of better usability. Anytime an organization sells stuff online, it's doing e-commerce. If you believe in your materials, you should want to sell more; the increased revenues will more than pay for the usability improvements.

Most non-profits accept donations on their site. Obviously, the actual donation pages should follow usability guidelines for registration and checkout. And beyond this? Non-profit sites are competing with many other places where people can spend their money, and such sites must be designed with this fact in mind. We're currently running an eyetracking study of the "About Us" pages for various charities, and we frequently hear users say that they don't feel like donating to a particular charity because the site doesn't present itself in a sufficiently credible manner.

Another example of how improved usability translates into economic value is in recruiting new staff. Last month, we tested sites for a study aimed at generating new material for our seminar on how an organization should present information about itself on its website. On the U.S. Department of Veterans Affairs (VA) site, we asked health care workers to look for jobs in the careers section.

Sadly, the main jobs page was rather confusing, with job information in three different places:

VA employment page as tested in 2007 (showing only the main content area in the center of the page). This area offered three different links for job information

Most users clicked on the top link, which is a typical response to an unclear set of choices. The top link led to a confusing site that looked as if it were recruiting police officers rather than health care personnel. The previous VA jobs page (above) had told job seekers to "simply" click on the "Agency Search" tab when they arrived on the "new" jobs page. Anytime you find yourself writing such instructions, you know you have a usability problem. Even worse, in this case, there was no such tab.

The destination for the first link on the VA jobs page, as tested in 2007 (showing only the top of the page). The page offered six tabs, but no "Agency Search" as promised on the VA site.

How many job applicants does the VA lose because of usability problems in its careers section? I don't know, but they lost one motivated participant in our study because he never found the openings that matched his specialty.

What's the value of getting more job applicants? Hard to say, but given that the VA's HR department has a recruitment budget, it obviously has some value. A simple way to get a rough estimate of job applicant value is to divide the recruiting budget by the current number of applicants. The resulting number is what your organization is paying to get a single resume. Fixing your careers site's usability could get you many more.

E-Government ROI Example

Let's perform a cost-benefit calculation for a specific usability problem.

The California Department of Motor Vehicles (DMV) lets people renew their car registration directly on www.dmv.ca.gov. This feature is advertised prominently in the annual renewal notice that's mailed to all car owners:

Unfortunately, the website's renewal page greets users with this warning:

Notice that the site insists that users have "a special RIN Number," which appears nowhere on the printed renewal notice. The notice does have a "Renewal Identification Number," but not everyone is going to decipher the acronym therein.

All experience shows that some users will be confused by the mismatched vocabulary, which violates one of the oldest usability guidelines: that of consistency. How many users? Hard to say without a big study, but I wouldn't be surprised if 2% of the users who arrive at this page leave it because they can't find their RIN numbers.

So, California has about 17.5 million cars. Let's say that 10% of owners try the online system for registration renewal. Given these assumptions, 35,000 people would abandon their online payment attempts because of the poor wording of the page's first bullet item.

Let's further assume that it costs $4 to process a paper payment and $2 to process an online payment. (For all these assumptions, the DMV would have exact numbers.) This means a cost of $70,000 per year for this usability problem.

Let's say that the site is redesigned every three years and discount future years' cash flows by 10% per year. Furthermore, let's assume that the percentage of car owners who use the online system grows to 15% in the second year and 20% in the third year. Under these assumptions, the net present value (NPV) over three years is $281,000.

What would it cost to fix this usability problem? First, the DMV would have to find it. A usability review costs $38,000 and identifies 68 usability problems on the average website. Clients tend to implement half of these recommendations. (The other half are either deferred because they're difficult to immediately implement or they're rejected because other business considerations are deemed to outweigh user experience.)

It therefore costs $38,000 to identify the 34 usability problems that will be fixed, or $1,118 per problem. The cost of an actual fix might be much more, but in our example, the cost is modest because all that's required is a bullet point rewrite. That is, the new text must be written, approved, and placed on the site. Let's say that all of this takes two hours, at a cost of $100 per hour, for an implementation cost of $200. The total cost to find and fix the usability problem therefore comes to $1,318. Because the improvement is worth $281,000, we conclude that the ROI is 21,220%.

An ROI of more than 21,000% is unusual, but not unheard of. Other usability issues will be harder to fix and thus have lower ROI; still, it's common to find ROIs of 1,000% or more.

Facilitating more online DMV renewals has a second benefit: every time people renew by mail, they place a burden on the environment. Let's say that the renewal letter weighs half an ounce (14 g). The 35,000 extra letters resulting from the poorly written bullet point will require the post office to truck half a ton of extra mail across California. The extra carbon emissions that result are certainly not going to help the governor meet his goals for terminating global warming. But the Web team could help by improving the site's usability.

Information-Only Sites

What about the many pages on government and non-profit websites that are purely informative? What's the value of improving the public information that organizations present?

At first glance, there's no monetary value in having users read more pages if those pages carry no advertising. However, a more indirect argument tells a different story:

1. Presumably, there's some value to the organization's activities. (If you don't agree, now would be a good time to quit your job.)
2. The value can be approximated by the budget. Congress, funding agencies, donors, or other financial supporters must believe that the organization's activities are worth what they cost or else they'd cut the budget.
3. The organization's website, published reports, and other information thus have a value that should be greater than the money spent creating this information, or the organization is mismanaging its budget.
4. Information has value only when it's being read and understood. In other words, the value we have imputed through steps 1–3 comes from having people read and understand the website.
5. Website use typically doubles when the site is made easier to use. Our studies of simplifying online information show that user understanding increases substantially when websites are rewritten according to usability guidelines. Lower-literacy users should be particularly important to government agencies, given their mandate to serve all citizens. These users' understanding increases the most when content usability is improved.

Thus, when we double the number of readers and increase their understanding of the website's content, we have doubled the value of having this information on the Web. That this value accrues to society at large should be fine with government agencies and non-profits, given their mission to benefit society.

Finally, answering the phone costs anywhere from $10–$100, depending on the call center staff's qualifications. The more people who can answer their own questions directly on the website, the fewer calls your organization will get. Reduced call center costs are often the most direct form of ROI from improving pure-information sites.

Usability ROI for Intranets

The ROI argument is the same for intranet usability, regardless of whether the intranet is in the commercial, non-profit, or public sector. In all cases, intranet usability leads to increased employee productivity, which is worth money. Save an hour of an employee's time, and you have saved the hourly cost of that employee.

Indeed, our latest Intranet Design Annual included two winners from the non-profit sector: National Geographic Society and The Royal Society for the Protection of Birds. In earlier years, other non-profit winners have included Luleå University of Technology, North Tyneside College, and the Mayo Clinic. Clearly, some non-profits are prioritizing good intranet design.

Non-profits staffed by volunteers might not see economic benefits from increasing the productivity of their intranet users. However, these non-profits should pay even more attention to intranet usability, because good volunteers are a precious resource. The more your volunteers feel that they're contributing to the cause, instead of struggling with bad computer systems, the more they'll be motivated to donate extra time to helping out. So, if you save an hour of a volunteer's time, you might earn two hours of work from that person.

Government agencies often have the best argument for improving intranet usability because they tend to be big organizations with many employees, meaning that the value of time savings multiplies into big numbers. For example, the U.S. Defense Finance and Accounting Service — which was one of our top government intranets — saved 200 staff years by improving its intranet's usability. Another winner, the U.K. Department for Transportation, saved £130,000 from a design change.

The ROI is There

Your project may not be commercial, but you're still spending money on your website and intranet and can add value by improving their designs.

Government agencies typically realize immense ROI from usability projects because they operate on a large scale with millions of website users and thousands or hundreds of thousands of intranet users.

Non-profit organizations are often smaller, but typically depend on donations, which they can increase substantially through user-centered website design.

It's a fallacy to believe that usability is only a concern for the commercial sector, just because that's where you find most high-visibility usability projects and hear tales of windfall profits from site improvements. The public sector and the non-profit sector also benefit immensely from usability, even if the calculation of benefits is sometimes slightly different.