← All checks
sitemap.xml checker: valid and listed in robots.txt
Group A3 pointsrubric r1
We fetch /sitemap.xml and look for a urlset or sitemapindex element, then check whether robots.txt has a Sitemap: line.
Why it matters
Without a sitemap an agent finds only pages that the homepage links to, so deep pages such as pricing details or documentation are missed. Two points come from a valid file at /sitemap.xml and one from the Sitemap line in robots.txt, which is how a crawler learns of a sitemap at another path.
How to fix it
- Generate a sitemap that starts with
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">. - Publish it at
/sitemap.xml, or make that URL redirect to the real file. - For many pages, publish a
<sitemapindex>that points to child sitemaps. - Add the line
Sitemap: https://example.com/sitemap.xmlto robots.txt. - List absolute canonical URLs and set
<lastmod>to a real date.
How to verify
Run curl -s https://example.com/sitemap.xml | head -c 300 and curl -s https://example.com/robots.txt | grep -i sitemap.