← All checks
Cloaking check: same content for bots and browsers?
Group E2 pointsrubric r1
We fetch your homepage as Chrome and as GPTBot and compare the length of visible text. A difference of 10% or less scores full points, up to 30% scores 1.
Why it matters
Serving different content to crawlers than to people is cloaking, and search engines penalize it. For an agent it means the text it quotes is not what the user sees when they click through. If the bot request fails outright the check scores 0, because the bot got no content to compare.
How to fix it
- Serve one HTML response regardless of the
User-Agentheader. - Remove rules that swap content, redirect or add banners for bot user agents.
- Keep personalization in the page, not in what the server sends to crawlers.
- Make sure the WAF does not return a shorter or error page to
GPTBot.
How to verify
Compare curl -s https://example.com/ | wc -c with the same command using -A "GPTBot/1.1"; the sizes should be close.