10 Reasons Why Google Isn’t Indexing Your Site
It doesn’t happen often, but when it does, it feels devastating. If you’re going to get any organic traffic from online searches, you need to make sure your site is visible—in other words, if you want to show up on Google’s search results pages, Google has to know that your site exists. And if your site isn’t being indexed by Google, it might as well not exist.
If your website isn’t appearing through organic search at all, fight the temptation to start panicking. Most of the time, this is simply an indication of some error or blockage that’s preventing Google from indexing your site—and these problems are easily fixed.
Take a look at these 10 reasons why Google might not be indexing your site—if you can’t be found in Google, chances are one of these is the culprit.
1. You don’t have both a www and non-www domain.
To the average web visitor, there’s no real difference between a URL that starts with http:// or http://www. Both of them ultimately lead you to the same place, so most users and webmasters don’t give it a second thought. But the www variant is actually a subdomain of the broader non-www version. In order to get your website indexed properly, you’ll need to verify your ownership of both in Google Webmaster Tools. You can also set your preferred domain, to inform Google which version you’d like to primarily use.
2. Google is still looking for your site.
If you’ve just launched a site and you excitedly scoured through Google to see your site listed, relax. It usually takes Google at least a few days to index a new site. If several days have already passed and you still haven’t seen any results, it could mean that Google is having trouble indexing your site—and that usually means you’re having an issue with a sitemap. If you haven’t yet created or uploaded a properly formatted sitemap, that could be your problem. Once corrected, you can “force” Google to crawl your website through Webmaster Tools.
3. You’ve got a lingering robots file.
Robots.txt files are shockingly common to find. Occasionally, developers or content managers will use a robots.txt file to prevent a search engine from indexing a given page. Essentially, the file communicates with Google crawlers and tells them not to index a site—so if you remove the file, you’ll cease to have an indexing problem. Do a thorough scan of your website code, and remove any instances of robots.txt files that aren’t there for a specific reason. You’ll still need to give Google a few days to index your site after correcting the erroneous file.
4. Google is experiencing crawling errors.
It doesn’t happen often, but there is a chance that Google is having trouble crawling some of your web pages. If your home page is indexing, but not all of your internal pages are, it could be a symptom of a simple crawling error. Log into Google Webmaster Tools and click on “Crawl,” then “Crawl Errors.” This will lead you to a list of any pages on your site that are currently experiencing crawling errors. These errors are sometimes attributable to robots.txt files, detailed above, but can also be the result of DNS errors or server errors, both of which are easily correctable in most circumstances.
5. Duplicate content is interfering with crawlers.
If you’re following best practices for content marketing, this shouldn’t be an issue, but there are circumstances where duplicate content can exist on your site—such as variations of a “master page” designed for slightly different audiences. If Google detects multiple instances of duplicate content, search engine crawlers can become confused and abandon indexing your site altogether. The easiest way to correct this is to get rid of the duplicate content. If deleting the duplicate content altogether isn’t an option, you can use 301 redirects or selective robots.txt files to ensure that Google only crawls one instance of each page.
6. Your site is experiencing loading problems.
If Google’s going to index your site, your site needs to be up. That means if you’re experiencing a loading problem when Google is attempting to index your site, you might miss the opportunity to be indexed. Ridiculously long loading times are sometimes the issue; if this is the case, you can decrease your loading times by setting up a decent caching system, reducing the size of your images, and installing a few applications to make the site run faster. It’s also possible that your hosting is unreliable, resulting in intermittent downtimes that are interrupting Google’s indexing attempts.
7. You’re using poorly optimized languages.
8. You’re being blocked by htaccess or privacy settings.
If you run a WordPress site, it’s possible you accidentally have privacy settings on—you can toggle this off by checking out “Privacy” under the Settings tab. It’s also possible that you’re using a .htaccess file for your website on the server. While .htaccess files are useful in most cases, they can sometimes interfere with site indexing.
9. You’ve got a Noindex or Nofollow indication somewhere in the meta tag.
Just like the robots.txt file, this is an addition that can mask your site’s pages from being found by search engine crawlers. Check your site’s code and look for the “noindex” tag somewhere in a meta title. If you find that somewhere, you’ve instantly diagnosed your indexing problem. Simply remove the tag and replace it if necessary, and you should be back on the fast track to search engine indexation.
10. You’ve been hit with a massive penalty.
When Google penalizes sites, it usually does so by dropping ranks and thus, visibility and traffic. However, there are rare and extreme cases when Google penalizes a site by completely removing it from indexes entirely. This is a type of manual penalty reserved for major infractions, so you don’t have to worry about this unless you’ve done something very wrong in the eyes of Google. If you’ve gotten deindexed this way, you’ve probably already been notified by Google, so unless that’s the case, you don’t have to worry that you’re not being indexed as a punishment.
Once your site is indexable, give Google a few days to catch up. You should start seeing your site in search engine results shortly. If you’re still having trouble, it’s possible your indexing problem could be more complex than usual. If you’re appearing, but you’re ranking very low, it could be an indication that your site is still new and doesn’t have much authority, or it could be an indication of a penalty. Either way, staying committed to best practices over an extended period of time is the best way to increase your visibility.
What can we help you with?
- Link building services for my company.
- White label link building for my clients.
- Major media brand mentions
- Something else (get in touch!)
Looking to grow your traffic?
Our managed SEO and social campaigns and high domain authority link building will increase your presence and organic search engine traffic.Request a rate card
Want more great resources?
Check out our new Resource Library, with over 100 expert articles spanning all aspects of online marketing, divided into 16 chapters.See our Resource Library