XML sitemaps: what to include, and how to split them

A sitemap does one thing: it tells search engines which URLs you consider canonical and worth crawling. It does not make pages rank, and it does not guarantee indexing.

Its real value for a small site is diagnostic. When Search Console reports "247 submitted, 189 indexed", the 58-page gap is the most actionable SEO number you have - and you only get it by submitting a sitemap.

What belongs in it

Only URLs that are all four of these:

1. Canonical. The version you want indexed, matching each page's own canonical tag.
2. Indexable. Returns 200, no noindex, not blocked in robots.txt.
3. Worth ranking. A real page, not a utility URL.
4. Genuine. No parameters, no session IDs, no tracked variants.

A sitemap full of URLs that redirect, 404 or carry noindex sends contradictory signals and degrades the trust Google places in the file.

What to leave out

• Paginated series beyond page one, unless each page has distinct content.
• Tag or filter pages with fewer than a handful of items.
• Search results pages.
• Login, cart, account and admin URLs.
• Any URL you have noindexed.
• Non-canonical duplicates.

The format

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://example.com/pricing</loc>
    <lastmod>2026-08-29</lastmod>
  </url>
</urlset>

loc is required and must be absolute. lastmod is used by Google - but only if it is accurate. A build process that stamps today's date on every URL every deploy makes the field worthless, and Google will start ignoring it.

changefreq and priority are ignored by Google entirely. Including them is harmless and pointless.

Limits and splitting

50,000 URLs or 50MB uncompressed per file, whichever comes first. Beyond that, use a sitemap index:

<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <sitemap><loc>https://example.com/sitemap/pages.xml</loc></sitemap>
  <sitemap><loc>https://example.com/sitemap/blog.xml</loc></sitemap>
</sitemapindex>

Split by content type long before you hit the limit. With one sitemap you know 58 URLs are unindexed. With separate sitemaps for pages, blog, tools and products, you know which type is failing - and that turns a mystery into a fix.

Getting it found

Two mechanisms, use both:

1. A Sitemap: line in robots.txt. Free, and it works for every engine.
2. Submit in Search Console and Bing Webmaster Tools. This is what gives you the coverage report, which is the entire diagnostic value.

Reading the coverage report

The indexed-versus-submitted gap tells you what is wrong:

• Over 90% indexed - healthy. Keep publishing.
• 70–90% - some thin or duplicate pages. Look at which type is failing.
• Under 70% - a real quality problem. Adding more pages will make it worse. Fix the existing ones first.

Common reasons for exclusion, in order of frequency:

• "Crawled - currently not indexed" - Google fetched it and decided it was not worth indexing. Thin content, or duplicative of something else on your site.
• "Discovered - currently not indexed" - Google knows the URL but has not crawled it. Usually a crawl budget signal on a large site, or a site with too few links.
• "Duplicate, Google chose a different canonical" - your canonical tag and Google's judgement disagree. Almost always a real duplication problem.
• "Page with redirect" - you put a redirecting URL in the sitemap. Remove it.