A sitemap is a file that lists the accessible URLs on a website. It comes in two main forms: the XML sitemap, intended for search engines, and the HTML sitemap, designed for human navigation. Understanding the structure of this file makes it possible to find specific information in seconds, as well as spot what’s missing or no longer working on a site.
XML sitemap and HTML sitemap: two files with different purposes
An XML sitemap is a technical file that can be read by Google and Bing crawlers. It contains a list of URLs along with metadata such as the date each page was last modified. This file isn’t intended for visitors: viewed in raw form, it displays lines of code.
An HTML sitemap, on the other hand, is a standard web page. It presents a site’s sections and subsections as clickable links, organized by topic. For users looking for specific information, the HTML sitemap provides quick access to different sections without having to use the site’s internal search engine.
When a sports website contains hundreds of pages (product pages, guides, categories), the complete sitemap lets you view the entire site structure and go straight to the section you want.
Spot missing or outdated pages with a sitemap
Most articles about sitemaps focus on how to find them or submit them to search engines. For a site manager, the most practical use is to compare the sitemap with the pages that are actually accessible.
A URL listed in the sitemap that returns a 404 error indicates a page that was deleted or moved without a redirect. A page missing from the sitemap, meanwhile, may never be crawled by search engine bots.
Check that the sitemap matches the live site
Opening the XML sitemap file (usually available at /sitemap.xml) gives you a complete list of the declared URLs. Comparing this list with the results from a crawling tool helps identify three problem cases:
- URLs listed in the sitemap but inaccessible (error pages, broken redirects, or content removed without updating the file)
- Pages published on the site but missing from the sitemap, making them potentially invisible to search engines
- URLs with an outdated lastmod tag, where the modification date doesn’t correspond to any actual change in the content
This last point deserves attention. Google may use the lastmod value to prioritize crawling recently updated pages. Changing this date without making an actual content update eventually makes the signal less reliable in the search engine’s eyes.
Metadata to ignore in an XML sitemap
The priority and changefreq tags still appear in many sitemap files. Google doesn’t take them into account. They have no effect on crawl frequency or page rankings. Only URLs and actual modification dates matter when processing the file.
How to find a website’s sitemap
Before you can use a sitemap, you need to find it. The paths vary depending on the CMS and the site’s configuration.
- The robots.txt file (available at /robots.txt) often contains a “Sitemap:” line indicating the exact URL of the XML file
- Popular CMS platforms (WordPress, Shopify, Wix) automatically generate a sitemap at standard locations such as /sitemap_index.xml or /sitemap.xml
- Google Search Console shows the sitemaps submitted by the site owner, along with the number of URLs detected and their indexing status
- As a last resort, searching Google with the operator site:domain.com filetype:xml may bring up the file
Sitemap limitations: what the file doesn’t guarantee
Submitting a sitemap to Google doesn’t guarantee that the URLs it contains will be crawled or indexed. A successful submission means the file has been read, not that the crawler will visit every page.
A syntactically valid file may list pages blocked by robots.txt, redirected URLs, or duplicate pages with a canonical tag pointing elsewhere. Technically validating a sitemap doesn’t replace checking the actual status of each URL individually.
For large websites, Google sets a limit of 50,000 URLs or 50 MB uncompressed per sitemap file. Beyond that, the file must be split into multiple sitemaps grouped under a sitemap index. An oversized or poorly segmented file may be partially ignored.
Whether XML or HTML, a sitemap remains a navigation and diagnostic tool. Used as a snapshot of a site’s structure, it reveals both the pages that exist and those that should exist. The next time a page seems impossible to find on a site, checking the sitemap before searching can often save time—or confirm that the content has simply disappeared.
