← All checks

Does your site block cloud and datacenter IPs?

We request the homepage with a browser user agent from our datacenter IP and fail the check on an error status or a bot-challenge page.

Why it matters

Agents and AI browsers do not run from home connections. Sites that serve a challenge page or a 403 to datacenter ranges show an agent an interstitial instead of content, and it reports that the site is unavailable. We detect the Cf-Mitigated header, "Just a moment" pages, and error pages that mention captcha or "access denied".

How to fix it

1. Review your WAF or CDN rules for blocks on hosting-provider ASNs.
2. Lower the challenge level so public GET pages are not challenged by IP reputation.
3. Skip bot challenges for static content, robots.txt, sitemap.xml and llms.txt.
4. Allowlist the IP lists that AI vendors publish for their crawlers.

How to verify

From a cloud VM run curl -sI https://example.com/ and confirm a 200 with no cf-mitigated header.

Check your site free

Scan a site

Related checks

• robots.txt reachable
• robots.txt parses
• GPTBot allowed
• ClaudeBot / anthropic-ai allowed
• PerplexityBot allowed

All checks