404 errors are silent killers. You may not know that a page on your site is offline until someone reports it or you happen to go looking for it—and by that time, you may have already lost significant traffic because of it. Imagine a user searching for a term relevant to your industry, and a deep page on your site pops up in the search results. They click the link, but it appears to be broken. In this scenario, they aren’t going to go out of their way to let you know. They aren’t going to come back later to check on it. They’re going to leave and will probably never come back.
To make matters worse, 404 errors are bad signs to Google. It indicates a previously existing, indexed page that is currently not available, which means your site is unreliable or your sitemap is inaccurate. You’ll drop out of the ranks quickly if the problem persists.
So how can you find 404 errors before they become a problem, and what do you do when you find one?
The easiest and most reliable way to find 404 errors on your site is through Google Webmaster Tools. After logging in, head to the Crawl tab and click on Crawl Errors. Under this menu, you’ll find a list of any web pages on your site that Google believes should exist (either because of a sitemap or because they previously existed) that can no longer be accessed by Google’s crawlers. If a page is inaccessible but still has an indexed URL, it will produce a 404 error upon being accessed.
There’s one caveat to using this tool, however. Because Google’s crawlers are far more technical and reaching than the average user (who will only encounter 404 errors via site navigation or search results), you may find a handful of 404 errors for pages that will never be encountered by an average visitor. For example, hidden links may be crawlable yet invisible to an average user. In these situations, it’s still a good idea to rectify the error—remember, your goal is to please both your user base and Google search crawlers.
Sometimes, a page may be down merely due to a technical issue. For example, if your blog is hosted separately, a server problem could easily cause a 404 error to display for your visitors. If this is the case, and the problem URL should theoretically be live and accessible, you’ll have to perform a root cause analysis to track down the source of the problem and correct it.
However, most of the time, a 404 error is the result of changing a URL or deleting an old page.
If the 404 error in question is an old webpage that you’ve long since deleted, or if it’s the old URL for a page that you’ve updated, the best way to correct the error is to create a redirect. Redirects tell users and web crawlers that the URL they’re trying to access is not directly valid, but instead should lead to a different page. This happens automatically on both fronts, so you never have to worry about either party coming face to face with a 404.
There are a few ways to do this, but the best way is with 301 redirects. Google has a helpful guide for developers trying to create their own redirects, but I’ll cover the basics here. Essentially, you’ll be reaching out to your hosting provider to create the necessary structures and traffic flows—for example, on Apache, you’ll be accessing your server’s .htaccess file.
Before you start to panic after finding a 404 error, it’s important to recognize that not all 404 errors are created equally, and not all of them are absolute emergencies. For example, if you have a hidden link that’s coming up as a 404 for a web crawler, but will probably never be encountered by a rogue user, you can afford to take your time in correcting the error. You might see a slight hit in your domain authority, but most sites have occasional (and small) 404 errors, so it shouldn’t interfere with your search visibility too much.
If your home page is appearing as a 404 error, or any other major page of your site, you need to take corrective action immediately. Such an error could instantly damage your reputation and cost you long-term customers as a result. As a general rule, the more visible the page with the 404 error is, the higher priority it is to correct the error.
Users generally don’t report 404 errors, so it’s up to you to perform periodic checks and ensure your site is functioning properly. How often you check is largely dependent on the size, complexity, and visibility of your site. For example, if you have a 10-page website that doesn’t get updated very often, you can probably get away with checking for crawl errors on a monthly basis (or even less often). If you have a 100-page website with new pages getting added and removed daily, it’s in your best interest to check for crawl errors every few days. It’s a good insurance policy to keep your site in good health and prevent your customers from seeing anything that might turn them away from the brand.