6% of the links on the Web are broken according to a recent survey by Terry Sullivan's All Things Web. Even worse, linkrot in May 1998 was double that found by a similar survey in August 1997.

Linkrot definitely reduces the usability of the Web, being cited as one of the biggest problems in using the Web by 60% of the users in the October 1997 GVU survey. This percentage was up from "only" 50% in the April 1997 survey. Users get irritated when they attempt to go somewhere, only to get their reward snatched away at the last moment by a 404 or other incomprehensible error message.

Even worse, linkrot contributes to dissolving the very fabric of the Web: there is a looming danger that the Web will stop being an interconnected universal hypertext and turn into a set of isolated info-islands. Anything that reduces the prevalence and usefulness of cross-site linking is a direct attack on the founding principle of the Web.

Most of this danger comes from attempts to use subscriptions instead of micropayments as a business model, thus erecting barriers to free navigation. Other dangers come from the craze for "portal" sites that guide users based on kickbacks instead of customer value: when links are determined by the size of payments instead of editorial judgment, users get cheated and benefit less from the Web. This is not to say that payments can't flow along links; just that these payments have to be generated by the users. Thus, it is fully ethical for Amazon.com to pay a commission to referring sites for books that users buy after following a link: the point is that payment only happens if the user is satisfied with the link and buys the recommended book.

Reducing Outbound Linkrot

Since users are irritated by linkrot, it is in your interest to reduce the amount of dead links in your own pages. The overall quality of the user experience strongly influences people's assessment of the credibility and value of a site: if a site doesn't work well, users will abandon it. Not only are dead links disappointing to users, they also rob your users of the value they were supposed to gain from going to the destination site. Remember, that one of the main reasons to include outbound links on a site is that users will credit you with some percentage of the value they gain from the sites you link to: thus, well-selected links enhance the value of your own service with the best of all the Internet has to offer, driving up user loyalty and repeat traffic to your site.

The standard advice is to run a link validator on your site at regular intervals. For small sites, it may make sense to outsource validation to a service that will spider your site maybe once a month and email back a list of dead links. For larger sites, it is more cost-effective to install validation software on the server itself. In either case, you need to have processes in place to contact the page authors to have them update or remove the offending links.

Let Incoming Links Live

Any URL that has ever been exposed to the Internet should live forever: never let any URL die since doing so means that other sites that link to you will experience linkrot. If these sites are conscientious, they will eventually update the link, but not all sites do so. Thus, many potential new users will be met by an error message the first time they visit your site instead of getting the valuable content they were expecting. Remember, people follow links because they want something on your site: the best possible introduction and more valuable than any advertising for attracting new customers.

Sometimes Web content becomes truly obsolete. An example would be the advance program and registration form for a conference that has already taken place. In such cases, it makes sense to remove the original page. Even so, the URL should still be kept alive and should be redirected to point to either a follow-up message (e.g., a report from events at the conference) or to a current page that is as close as possible to the original one (e.g., the program for next year's conference).

At other times, it becomes necessary to re-architect a site and impose a new structure. Even then, the rule continues to be: you are not allowed to break any old links. The solution is to set up a set of redirects: a scheme whereby the server tells the browser that the requested page is to be found at a new URL. All decent browsers will automatically take the user to the new URL, and really good browsers will even update their bookmark database to use the new URL in the future if the user had bookmarked the old URL.

Any time one of your old URLs stop working, you are throwing away business. It is like refusing entry to a shop for anybody who is dressed in last year's fashion.

  • Keep all old pages on your server forever (unless they are truly misleading and are replaced by an update)
  • If moving pages, leave a redirect behind