What is a sitemap? A guide

A sitemap is a roadmap to your website. It gives search engines and users easy access to its key pages and content.

They can be vital when it comes to SEO success, but how exactly do they work? Why do you need one, and how do you go about creating one for your site?

Our ultimate guide to sitemaps answers all of these questions and more. You’ll find the following topics covered below:

• What is a sitemap?
• What are they used for?
• Do I need one?
• When were they introduced?
• Sitemap anatomy
• What different types of sitemap are there?
• Creating a sitemap
• How to submit it to Google
• How to test a sitemap

What is a sitemap?

A sitemap is a complete map of your website that allows search engines and visitors to find their way around its pages and content. They can broadly be divided into two types: XML sitemaps and HTML sitemaps.

An XML sitemap is basically a file that stores information about the pages, videos and other content on your site as well as the relationship between them. All of this is based on the data you add to the file yourself.

When there’s an XML sitemap in place, search engines can crawl a website more intelligently, learning valuable information about it along the way. This includes when it was most recently updated, how often its content is refreshed and whether alternate language versions of the site are available.

HTML sitemaps, meanwhile, are visual tables of contents aimed at site visitors, rather than search engines. Their main purpose is to help users navigate your website, but they can have SEO benefits, too.

From here on in, this guide mainly focuses on XML sitemaps. 

What are sitemaps used for? 

They allow webmasters to signpost the most important pages and content on their website and relay this information to search engines. This gives Google and its competitors a clearer idea of your site’s structure and insight into the nature of the content it hosts.

This includes fine details about the video, image and other media files on your website. For instance, a sitemap video entry can tell Google how long the clip runs for, what category it belongs-to and whether there’s an age-appropriateness rating for users to be aware of. An image entry, meanwhile, can shed light on its subject matter, file type and usage rights.

Giving this key information to search engines to help them better understand the context of your website and how to categorise it. This can give your site an SEO boost.

Do I need one? 

Sitemaps can be useful if your site is particularly large. Like if it has an archive of pages that aren’t linked well, or contains a vast amount of media files. That said, Google themselves have confirmed that “a sitemap can improve the crawling of your site” in most cases.

There are some situations where a sitemap is more important that others. But the majority of websites would benefit from having one, and you’ll never be penalised for it.

When were sitemaps introduced?

Google introduced sitemaps back in the summer of 2005 to help web developers make their sites more crawler-friendly by publishing a comprehensive list of links. The web giant launched a joint initiative with Yahoo and Microsoft the following year to support Sitemap 0.90 – the next schema version of the web tech. They encouraged its widespread adoption.

By April of 2007, Ask.com and IBM had added their names to the list of sitemap supporters, while Google, Yahoo and MSN added auto-discovery for sitemaps to their search engines. With Google’s help, the state governments of Arizona, California, Utah and Virginia were among the first major organisations to add sitemaps to their websites later that year.

In 2020, there was another milestone in the rollout and widespread usage of sitemaps. Thanks to a collaboration between developers at Google and Yoast, XML sitemaps were added to the core of market-leading content management platform WordPress.

A bespoke plugin entered testing in late January before the feature was rolled out as part of the WordPress 5.5 update, allowing users to add native Google sitemap integration directly from their content management systems.

The anatomy of sitemaps

Below, you can see what the raw code for a standard XML sitemap entry looks like:

<url>
<loc>http://www.hypothetical.com/webpage</loc>
<lastmod>2020-09-09</lastmod>
<changefreq>monthly</changefreq>
<priority>1</priority>
</url>

The code above relays key information to search engines:

• where this page is located (loc)
• when it was last updated (lastmod)
• how often its content is likely to change (changefreq)
• how important it is in the context of the website (priority)

Changefreq and priority, in particular, are important tags as they can determine when and how often search engine crawlers visit your website, which can have far-reaching implications.

If you’re creating a sitemap manually, you’ll need to master the use of these tags. However, there are a range of tools available online that will fully automate the process (see the ‘How to create a sitemap section’ for more information on this).

While sitemaps like these give search engines the data they need on your website’s pages, you’ll also need to learn the anatomy of specific supporting XML sitemaps. That’s to provide them with key info on your images, videos, mobile and news content.

Supporting XML sitemaps include:

• Image XML sitemaps: For relaying information about your website’s images and guiding search engines to the pages they’re located on.

• Video XML sitemaps: These provide search engines with key information about the clips you’re hosting, like running time and age restrictions.

• News XML sitemaps: Help search engines pick up any news content you might be hosting. Details might include the language it’s in and access information.

• Mobile XML sitemaps: Many websites now optimise their content for tablets and smartphones. With these supporting sitemaps, you can let search engines know exactly where your mobile-friendly pages are located.

What different types of sitemap are there?

As we’ve already touched on, they’re usually split into two types: HTML sitemaps and XML sitemaps. Both can bring SEO benefits, but there are important differences between them that you’ll need to grasp.

Let’s start with the HTML variety, since we haven’t explored them in any detail so far. These are mainly useful for visitors to your site, offering them clear pathways to navigate its pages and content. A HTML sitemap is basically a hub that lays out your site’s table of contents in a visible format for the user. Adding Cascading Style Sheets (CSS) can make the experience more visually appealing for them.

As well as boosting your site’s user-friendliness, having a HTML sitemap can improve your website’s SEO, but their XML counterparts make this a bigger priority.

Indeed, XML sitemaps are only used by search engines to crawl the contents of your site. They’re not meant to be visible to site visitors since they take the form of an xml file placed in the website’s root directory.

Not only do they provide a roadmap for Google and co. to follow, they contain specific information about your website. For example, how often it’s updated. So search engines can understand it contextually and rank it appropriately.

Within the broad XML category, there are also supporting XML sitemaps for news, video, image and mobile content (see the previous section for more on these).

How to create a sitemap

Creating a sitemap manually involves a lot of legwork. To start, you’d need to thoroughly review the structure of your site and draw up a template of its pages and content. Then create a hierarchy of pages based on how you want them to be indexed.

Next, you will need to manually code your URLs and validate the code before adding the sitemap to the root and robots.txt. Not only does this call for a degree of technical know-how, there’s also the margin of error to take into account.

Luckily, there are handy tools that can take the pain out of the process by generating and checking your sitemap for you. The 20i Sitemap Generator is an example of such tools.

It’s a simple one-click tool that uses a bot to crawl your website. It will create a sitemap of accessible pages automatically, obeying all robots.txt restrictions along the way. What’s more, you can set it to automatically update your sitemap at preset intervals to make sure your latest content is listed.

How to submit a sitemap to Google

Once your sitemap is firmly in place, you’ll need to submit it to search engines. The easiest way to do this is through Google Search Console. If you’ve never used Search Console before, it’s straightforward to get started with it through the link.

Once you’re on the dashboard, select ‘Sitemaps’ within the ‘Index’ section of the side menu. Then enter the URL (e.g. yourwebsite.com/sitemap.xml).

It’s obviously a good idea to sweep your sitemap for errors earlier in the process (see the next section for more on this) as Google takes over once you’ve clicked that ‘Submit’ button.

Once you’ve submitted your sitemap, crawlers should be able to index your website with greater ease and accuracy. In turn, it could lead to gains in the search engine results.

How to test a sitemap

Human error is possible if you’ve coded it yourself, and this could mean that your sitemap doesn’t function the way it’s supposed-to. There are tools available online to help you validate your sitemap’s code and make sure there are no syntax gaffes.

One of the most popular sitemap checkers is the XML Sitemap Validator. This automatically sweeps your sitemap for errors when you input its URL into the system. Something as simple as a missing tag is enough to throw a spanner into your website’s works. This application seeks out these errors so you can fix them quickly.

Alternatively, once you’ve submitted your sitemap in Search Console you can see errors by clicking on ‘See Index Coverage’ in the right-hand column. You’ll then get a report. This is useful as a last-minute safety net, though seeking out errors before this point is recommended.

Hopefully that covers everything you need to know about sitemaps. Let us know if you have any questions!