A (Nearly) Comprehensive Guide to On-site OptimizationLeave a Comment
So you’re into SEO and you’re ready to get your hands dirty.
You might be starting a campaign for the first time, gathering some preliminary research, trying to fix something that’s wrong in your strategy, or just brushing up on your skills. In any case, this is the guide you want.
As you probably already know, SEO is split into two main worlds: onsite and offsite optimization.
Onsite optimization refers to all the structures, techniques, and strategies necessary to include on your website, including all your individual pages. Offsite optimization refers to everything that happens outside that world, including links pointing to your site, social media activity of your brand and your users, guest posts, and so on.
This guide is exclusively about onsite optimization. I’ll be writing more about offsite optimization in future guides, but first, I want to explain the essence of onsite optimization, why it’s important, and what you need to do to be successful.
As I mentioned above, onsite optimization is about what you do on your own site. Onsite optimization can be broken down into individual tactics, all of which cumulatively impact your site’s visibility in search engines. Unfortunately, it’s not as simple as flipping a switch, or adopting one new habit—there are many different tactics you’ll need to adopt, in many different areas.
For starters, there are onsite optimization tactics that can be done only once—when you commit the change, you won’t have to worry about it again (at least until something changes or breaks). Others require ongoing attention. Some are structural, impacting the design and layout of your site, while others are qualitative, subjectively evaluating certain elements of your site.
I’ll be posting a quick-reference checklist of all the onsite optimization factors you’ll need to consider at the end of this article, but because these tactics are so diverse in nature, I want to make sure you understand the theory behind them as much as their raw implementation.
This guide is broken down into three main categories of tactics and techniques:
- Indexation is your biggest priority, because it’s what makes your site visible to Google in the first place. If Google can’t index your site, or if it isn’t indexing your site properly, your pages won’t appear in any search results (or might appear incorrectly).
- Being seen is important, but you also want to make sure your site is understood the right way. Including the right content, including titles, descriptions, and body copy of your pages, can ensure Google properly categorizes your site and presents it for relevant searches.
- How your site functions, displays, and how your users interact with your site are also important considerations. It’s in Google’s best interest to rank sites with high performance levels, so make sure yours is functioning properly.
Let’s work on exploring each of these categories individually.
Think of Google as a massive library that offers books for people searching for various topics. The first step to getting your book found is making sure your book is on the shelf—so let’s get it there.
Your first job is to make sure that Google’s web crawlers can access your site. Think of these bots as scouts that work on Google’s behalf to scour the web and index information. If these crawlers can’t see your site or can’t access it, Google won’t be able to index it.
There are a handful of reasons why this might be the case:
- There’s a server-side error preventing bots from reaching your site.
- Your site is down or otherwise inaccessible to any user.
- You’ve accidentally blocked web crawlers in your robots.txt file (more on this below).
It’s also worth mentioning that there are multiple web crawlers out there—several specific to Google, and several belonging to other major search engines and tech companies like Bing and Apple. Here are some of Google’s most relevant ones:
Unless there’s something inherently wrong with your site or server, your site should be crawlable. It’s actually harder to stop Googlebot (and other search engine bots) from finding your site. If your site is new, it might take a few days to a few weeks to make it to Google’s index, so don’t be alarmed if you aren’t popping up in search results.
Your robots.txt file is like an instruction manual you can post to search engine bots in your top-level directory. It tells them which pages they should crawl and index, and which ones to avoid. By default, web crawlers index the entirety of your site, but there may be certain pages you don’t want indexed (e.g., pages with duplicate content).
Before doing anything on your site, a bot will check the reference:
This will specify a User-agent and specific pages with a Disallow tag. With the User-agent specification, you can exclude specific bots (see table in previous section) or reference all bots. The Disallow feature will then allow you to exclude any pages you don’t want to be indexed.
As a general rule, you only need to worry about this if you have canonical issues to resolve, or if there’s a page that might interfere with your primary SEO goals. Otherwise, you can leave your robots.txt file blank. Either way, double check your work to make sure you haven’t accidentally precluded all search bots from seeing your entire site—it happens more often than you might think.
One word of advice: don’t try to be sneaky by hiding bad or damaging material. Robots.txt instructions are publicly available information. You can see ours at AudienceBloom here:
If you’re concerned about the formatting or function of your robots.txt file, you can use Google’s free tester to check it out for possible errors.
Your URL structure can influence how your site is seen and how your pages are evaluated. Google favors sites that have clear, straightforward URLs that make it easier for users to navigate, along with descriptive text that tells Google what the page is about.
- Avoid dynamic URLs. This makes it confusing for Google’s index and could be an indication of a deceptive practice.
- Avoid any special characters or long numerical sequences, like “&$%^*” or “321987662090.”
- Include a “breadcrumbs” trail, demonstrating the location of each page within subpages and categories. For example: Domain.com/first-category/secondary-category/final-page
- Separate words with dashes “-“ rather than underscores “_”
- Keep your URLs as concise as possible.
- Include clear, descriptive text at the end of each URL, preferably including your target keywords.
Take this link as an example: http://www.audiencebloom.com/2016/01/how-to-tell-if-an-seo-agency-is-trustworthy-in-5-steps/
It’s static, concise, offers a breadcrumbs trail for the blog, and accurately describes the content of the page with relevant keywords.
There are actually two kinds of sitemaps you can offer for your site, and both are important for SEO. How important? That’s somewhat debatable, but it’s almost certainly worth the effort to create.
HTML sitemaps exist for users and search engine crawlers, and can usually be found in the footer of a website. It’s a good idea to make sure every page of your site links to this, so having it in the footer is the fastest and most reliable way to establish this. You can find ours here.
XML sitemaps are a bit more technical, and you can upload them directly to Google via Webmaster Tools.Just head to the “Sitemaps” section and click “add/test sitemap” in the upper-right hand corner.
If there are any specific issues with your sitemap, Google will let you know.
Here’s a great example of one given by Sitemaps.org (an ideal resource for understanding more about XML sitemaps):
Keep in mind that your site is always changing—you’re almost constantly adding pages, removing pages, or changing pages, so work to keep your site maps up-to-date. If you need some additional help, there are many popular site crawlers available online—one of the most popular is Screaming Frog, which is free for up to 500 URLs.
Do you see anything like this on your website?
That’s bad. All your content should be able to load properly on any device, with any browser, on practically any speed Internet connection. Your content should load directly from HTML (you don’t have to avoid AJAX or iFrames altogether, but the bulk of your content should come from HTML directly), and return no errors when user access is attempted.
The reason for this should be obvious. Google wants to give people actual content—not blank spaces where content should be. Even if it wasn’t a search ranking factor, it would be an important user experience factor, so don’t neglect it.
If you’ve done any significant searching in the past few years, you’ve probably come across something like this:
Note the phrasing of the question and the purported answer, sectioned off from the rest of the search results. This is known as a “rich answer,” and it’s a part of Google’s Knowledge Graph. The Knowledge Graph isn’t a bank of information so much as it’s a network that taps into information on other websites. In this case, my query “how many US citizens are there” prompted Google to find the answer on the Wikipedia page “Demography of the United States.”
Unfortunately, Google can’t do this all on its own—it needs help from webmasters to properly categorize and submit information. For webmasters, this presents a valuable ranking opportunity—it won’t increase your domain authority, but it will give you the chance to have your information posted prominently above traditional results.
The way to categorize your information is through microformatting, sometimes referred to as structured data or structured markup. Basically, it’s a coding format you can use on your site to tell Google how to read information like events, people, organizations, actions, reviews, and many other archetypes. Since it gets technically complex (and warrants an article of its own), I won’t get into the details here, but Schema.org is a leading authority in microformatting, and offers detailed information how to apply it to your site.
Google Analytics and Webmaster Tools
This technically isn’t going to help your ranks—at least not directly—but signing up for Google Analytics and Webmaster Tools is essential if you want to gain more knowledge about your site, proactively respond to pressing issues, and learn how your strategies are working. If you have a Google account, you’re already halfway there. Google Analytics will prompt you to create a new site and place a tracking script in your code, and Webmaster Tools will require you to verify your ownership by putting a short verification script in your code or verifying your webmaster’s email address.
I’ve already mentioned some of the onsite insights these tools can offer you, such as crawling your site and submitting a sitemap, and I’ll mention more, such as scouting for duplicate content and evaluating your meta data, but know that there are many more features to explore to improve your site.
Optimizing Individual Pages
Now that you know your site is properly indexed, let’s work on optimizing the individual pages of your site. You’ll have to apply these changes to each page of your site, so be sure to implement them for every new page as you add them.
Page Titles and Meta Descriptions
Let’s talk about titles and descriptions. Check this out:
The above example is a search for “AudienceBloom” and naturally, we’re the first to appear. Take a look at the sections of the entry highlighted above. The headline, with the link embedded, is the title of this page, while the short description below it is the description or “meta description.”
Titles and descriptions play two main roles in the SEO world:
- They tell Google what your content is about. For example, if your title is something like “Why dogs bark at cars,” Google will understand the purpose of your page and will be more likely to feature it for a search like “why is my dog barking at cars.”
- They form users’ first impressions when they see your site in search listings. It’s important to start strong and entice users to click.
Accordingly, your titles and descriptions should both exhibit the following qualities for all pages:
- Unique. Don’t duplicate titles or descriptions, even if you’re trying to save time. You can pull up a list of duplicate meta data entries in Google WMT by heading to Search Appearance > HTML Improvements.
- Concise. Don’t try to say more than you have to.
- Descriptive. Describe your content as accurately as possible, including keywords relevant to your material. However, don’t stuff your titles or descriptions with keywords; only use them as you would in a natural conversation. Keyword stuffing is a big no-no.
- Branded. However, your branding should appear toward the end of your titles for most pages, such as a “Primary keyword phrase and title | Brand name” format.
- Compelling. Remember, you’re doing this to please users as much as you are search engines.
With all that in mind, what makes titles and descriptions different?
Just a few things:
- Titles are more important. This is true for both search engines and users, so make sure your titles are as good as they can be. You have more wiggle room with descriptions.
- Each has different length requirements. There’s a firm character limit, and if you pass it, Google will truncate your material (something you want to avoid if possible, but it’s not the end of the world if you have occasional discrepancies). The limit for titles is 75 characters, while descriptions get 160.
While we’re talking about titles and descriptions, don’t forget your header tags. Numbered in sequence (H1, H2, H3, etc.), header tags indicate the main points of your body content—almost like a table of contents. This can help search engines properly understand and index your content, as headers are weighted more heavily than standard body copy.
I’ve already gone over what makes a good URL (in the Indexation section above), so I won’t repeat myself. However, it’s important to remind you that each page should have a properly formatted URL, preferably under 90 characters. Keep this in mind whenever adding a new page.
Your on-page content tells Google much about your page. Though it usually serves as supplementary information to the more-important titles, descriptions, and headers (see two sections above), you shouldn’t neglect the on-page content for any page of your site. At a minimum, you should have 100 words of highly descriptive content. If you can’t offer that, you probably should have a page here.
Content gives you three opportunities:
- It shows you care about your audience. Google will only rank pages with sufficient content because otherwise, users won’t get what they need.
- It gives you an opportunity to showcase more keywords. In addition to the keywords included in your title, description, and headers, you can use variations and synonyms of your keyword phrase to capitalize on semantic search patterns. Again, be careful not to over-optimize; only use terms as they would appear naturally.
- It can help you earn links. Inbound links mean greater authority, but external sites will only link to you if your content is valuable.
There are many factors for what’s considered “quality” content—far too many to list here, but these basics should get you started in the right direction.
A Note on Duplicate Content
One quick note—all content on your site should be unique (meaning it doesn’t appear anywhere else on your site). Sometimes, alternative URL forms (like http:// vs. https://) can cause Google to index one page twice over and register that as duplicate content. This is bad news. Fortunately, it’s easy to detect and correct—take a look in Google WMT under Search Appearance > HTML Improvements and you can generate a list of duplicate content instances. From here, you can either use your robots.txt file (see above) to block one instance of each occurring offender, or set up 301 redirects to properly canonicalizeyour links.
It’s a good idea to include images wherever you can on your site. In combination with high-quality written content, these help convey to Google that you’re a high-authority site dedicated to bringing great content to your users. However, you can’t just stuff images all over your site and expect to rank higher.
There are two main ways images can increase your search visibility:
- Their contextual relevance signals quality and context to Google. For example, if you title your image “example of poison dart frog in Amazon rain forest” in your article “Poisonous animals to watch for in the Amazon,” Google will have a clear picture of the content you’re trying to offer users.
- Their direct descriptions makes them capable of appearing in Image searches. Image search isn’t as popular as traditional searching, but you never know when someone might be searching for “dart frog” pictures and end up stumbling on your site—it’s an easy opportunity to capture, so don’t miss it.
There are two ways to optimize your images:
- Titles. Titles should be contextually relevant to your piece and your image.
- Alt tags. Alt tags serve as direct descriptors of your image content.
For example, take this picture of the Washington Monument:
A good title might be: “This Washington Monument photo illustrates how to optimize an image for SEO”
While good alt text might be: “Washington Monument against sky”
Notice how I’m not stuffing either of these with keywords, nor am I describing something that isn’t there. Something like “Monument SEO best practices and onsite optimization” wouldn’t serve me well (and probably doesn’t serve me well in the body of this paragraph, either).
In addition, your titles and alt tags should follow most of the general best practices I outlined for page titles and descriptions (namely unique, concise, descriptive, and compelling).
You can also optimize your images by making them a proper format (.jpg and .gif are popular standbys) and by making them smaller and easier to download (maximizing site speed—more on that later).
Internal Links and External Links
Most of your onsite content should include links to other pages, both internal and external.
Internal links are important because they establish connections between different areas of your site and make it easier for your users to navigate. The more tightly linked your site is, the happier Google will be. As a general rule, no single page of your site should be more than four clicks away from any other page at any other time.
External links are important because they show you aren’t just making things up—they’re your callouts to outside authorities.
For both types of links, it’s important that your anchor text is accurate and descriptive; don’t just name the page you’re linking to, and don’t try stuffing your anchor text with keywords.
How your site performs can also factor into your domain authority, which in turn influences how your pages rank in searches. These are generally secondary to factors like your site structure and onsite content, but can influence your final ranks.
Mobile optimization isn’t optional. Last year, mobile traffic overtook desktop traffic for the first time, and it still hasn’t stopped growing. If that wasn’t enough incentive, Google then released its “Mobilegeddon” update to reward every site with a fully functioning mobile site and punish those without one.
Optimized for mobile means your site:
- Loads quickly and easily on mobile devices.
- Offers a design that doesn’t require zooming in and out, in a “thumb-friendly” format.
- Presents text in a readable format without the need to zoom.
- Has all its images and videos viewable on mobile.
- Has mobile-friendly buttons and form-fields.
There are a few ways to achieve this, but by far the easiest and most popular is through responsive design. Responsive design automatically flexes a site to accommodate any device that accesses it.
A good example is CNN’s site:
Look how the content is basically the same, but “stacked” on the mobile version to make it easier for mobile users to access. If you’re in doubt about whether your site is considered “mobile-friendly,” Google offers a free test you can use to find out.
Uptime and 404 Errors
Your site should be up most of the time. If it ever goes down, due to a server issue or maintenance, you should be aware of it and work quickly to restore it to normal. This should go without saying.
404 errors offer a bit more flexibility; these come into existence when one of your pages no longer exists (usually because it was deleted, renamed, or moved). 404 errors don’t hurt your ranks directly, but they can cause you some user experience woes—for example, if a user follows an old link or sees an old indexed page but only finds a 404 error, they may leave and never return.
There are two easy ways to “fix” a 404 error, and both are welcomed by Google:
- Restore the page. Fix the naming convention, or put the page back where it was.
- Create a 301 redirect. Redirect traffic to the new version of the page, or a similar page. You can read more about how to do this here, straight from Google.
That being said, there are some instances where leaving a 404 error alone is the best option, such as when the page is simply no longer relevant to your brand.
Site speed isn’t as big of a ranking factor as some might claim, but it can influence your authority. The faster your site loads, the happier your users will be—giving you a ranking bonus as well as a brand reputation bonus.
There are several ways to speed up the performance of your site, including:
- Reducing the size of your images.
- Deleting plugins or add-ons you don’t need.
- Using a caching plugin (effectively).
- Enabling compression.
- Deleting old or unused drafts of content.
- Correcting any hosting problems that could be holding you back.
Site speed is especially important for mobile users, as most mobile devices offer slower loading times than comparable desktop connections. Mobile users also tend to be more demanding, so every second here counts.
Keeping your site secure won’t give you much of an extra ranking boost, but it will be valuable to your users. Opt for SSL encryption (you can tell you have this by the “s” in https://), and your users’ data will be more secure. Still, https is a ranking signal, and it may grow in power as the years go on. You can purchase an SSL certificate through your hosting provider.
A Note on CMSs
Before concluding this article, I want to mention something about CMSs. Most modern CMSs, including the ever-more-popular WordPress, offer built-in SEO features, some of which claim to optimize your site on your behalf or “automatically,” and others of which present these options in easier interfaces, such as allowing you to type in your titles and descriptions rather than embedding them in code.
Most of these tools are valuable time-savers, helping you reduce your margin of error and get your work done faster and more efficiently. However, don’t make any assumptions. It’s not enough to assume that your CMS “took care” of something for you. Run the tests yourself and don’t be afraid to dig into the code of your site.
Your Ultimate On-Site SEO Checklist
I’ve given you a lot of information, so to make things easier, here’s an “ultimate” checklist you can use to make sure you’re optimizing your site effectively (split into sitewide and page-level sections).
Print it out and keep it handy:
- Double check and upload your robots.txt file to ensure search engines index your site properly.
- Check to note any server errors that could prevent indexation.
- Keep all your URLs static, organized logically with a breadcrumbs trail and minimal characters.
- Use an HTML and an XML sitemap, and keep them updated.
- Ensure your content loads properly, straight from HTML, and on all devices and browsers.
- Use microformatting to feed search engines structured information.
- Use internal links with descriptive anchor text to make your site more navigable.
- Use external links with descriptive anchor text to make your site more authoritative.
- Ensure all images are equipped with unique descriptions and alt tags.
- Maximize site speed on all pages.
- Ensure site security with SSL encryption.
For Individual Pages:
- Make sure your title tags are under 70 characters, unique, descriptive, and include a reference to a target keyword.
- Make sure your meta descriptions are under 160 characters, unique, descriptive, and include a reference to a target keyword.
- Include proper H1, H2, H3 (and so on) header tags on all your articles.
- Keep each URL under 90 characters, with descriptive text and no strange characters.
- Include a few hundred words (minimum) of descriptive, unique content on every page.
With that, you should have everything you need to get started with onsite SEO (or check to ensure you’re doing everything properly). If you can cross off all the items on this checklist for all of your pages (and keep that quality consistent as your site grows and changes), you can consider your onsite work nearly complete. Beyond these factors, your greatest concerns should be user experience and ongoing content quality—but those are topics for another post.