You’ve done all the upfront work for your SEO campaign. You’ve carefully designed your navigation and internal pages so that your users have the best possible experience. You’ve fine-tuned your onsite copy and page titles to be optimized for searches. You’ve even spent the last several months updating your blog with fresh, authentic, well-written posts to attract new customers and show Google that you really know your stuff.
There’s only one problem: if Google doesn’t see what you’ve been doing, it doesn’t matter how much you’ve done. If there’s some kind of blinder preventing Google from being able to see or crawl your material, it might as well not even exist.
Fortunately, Google knows how destructive these blinders can be, and it knows that they’re sometimes undetectable (since Google’s algorithm operates invisibly in the background). In response, it’s developed a series of tools you can use to test whether there are currently any page errors interfering with the normal crawling of your site.
The first thing you’ll need to do is set up a Webmaster Tools account (if you haven’t already). Here, you’ll be able to access all these tools at any time and determine the state of your site.
There are many different factors that could lead to Google being unable to see or index your site, and all of them are equally bad. However, some of them are more complex to fix than others. Knowing where to look for these errors is half the battle; the other half is simply a matter of correcting them once they arise.
Overall Site Errors
If you’re paying attention, you should be able to catch broad site errors on your own. Still, sometimes your entire site can go down and it’s a matter of days before you notice. This is bad news. In Webmaster Tools, head to the “Crawl” tab and check out “Crawl Errors.” At the top, you’ll see a short row of different statuses, including “DNS,” “Server Connectivity,” and “Robots.txt Fetch.” A little green checkmark next to each of these will let you know that your site is up and running.
If your site is down, one of these will likely be responsible for the issue. You can use this diagnostic report to determine how to take corrective action.
Individual Page Errors
You’re going to stay in the same place for these type of errors, which are far more common and less noticeable. Scroll down to where you see the phrase “URL Errors.” Here, you’ll find data from the past 90 days on all the individual pages of your site that are returning errors. Sometimes, this is due to loading issues or other minor forms of interference, but for the most part, these will be the all-too-common “404 error” or “Not found” error.
Even if you’re generally on top of your work, you’re bound to have at least a handful of your internal pages showing up here as 404 errors. This can be due to changing your page navigation without updating your sitemap, changing the URL name of a page without setting up a redirect, or just taking a page down and forgetting to make the necessary updates afterward. In any case, this report will show you exactly which pages on your site are returning the error and when the initial instance of the error was detected.
The best way to fix these is to either restore the pages in question or set up a 301 redirect, which will direct Google’s bots to head to a new URL instead of the old, problematic one.
Meta Data Errors
These aren’t actually errors, per say, but they can get in the way of your site’s ranking and are notoriously hard to detect without the proper toolset.
In Webmaster Tools, head to the “Search Appearance” tab and click on “HTML Improvements.” Here, you’ll find a list of different discrepancies with your site’s meta data—namely, the title tags and meta descriptions of your individual pages. This list will be conveniently broken out into which are duplicates, which are too long, which are too short, and which are non-informative.
There’s no easy way to fix all of these errors at once. Instead, you’ll have to take a look at an individual page level and rewrite the titles and descriptions under scrutiny. With duplicate meta data issues, sometimes the problem is fixed as easily as changing a couple of words.
Finally, you’ll want to check that Google is properly indexing all your pages. In this process, you’re going to be comparing two things: the first is your own sitemap and the second is the “index status” you’ll find in Webmaster Tools. You can find this under “Google Index” and “Index Status.” Under the Basic view, you’ll be able to see how many pages Google is currently indexing—if this number doesn’t match the number of pages on your sitemap, you have an indexing problem. Check to see that your sitemap is up to date, and that none of your pages are currently being blocked by robots.txt. If you have any recently added pages, remember it may take a few days before Google indexes those pages.
Page errors can seriously slow down your SEO efforts, but they’re only temporary setbacks. Figure out where the problematic areas are, take corrective action, and it won’t be long before your rankings are back to normal.