Cloudflare SEO checker: is Cloudflare blocking Google or AI crawlers?
Since 15 September 2026, Cloudflare's "Block" setting for AI training also applies to Googlebot, Bingbot and Applebot. One toggle can take a site out of search without any change to robots.txt or your CMS. Check what Cloudflare does with each crawler.
What this check covers
Edge vs origin
Tells a Cloudflare challenge or block apart from an error from your server.
Googlebot + AI pattern
Flags the signature of an AI training block hitting Googlebot.
Bot Fight Mode
Detects managed challenges served to crawlers behind a 200 or 403.
Read-only settings check
With a Bot Management: Read token we read AI Crawl Control and Bot Fight Mode directly.
Sitemap as Googlebot
The first symptom reported in August 2026 was a 403 on the sitemap.
Daily monitoring
Alerts when a settings change starts blocking crawlers.
Questions
Does Cloudflare block Googlebot?
Not by default: verified search crawlers are allowed. But "Block" for AI training applies to mixed-use crawlers including Googlebot since 15 September 2026, and custom WAF rules or bot settings can block it too.
How do I stop AI training without losing Google?
In Cloudflare → Security → AI Crawl Control, set Training to "Disallow AI Training" instead of "Block". It publishes a no-training signal in robots.txt and keeps search crawlers allowed.
Is it safe to give you a Cloudflare token?
Create a token with only Zone: Read and Bot Management: Read for the zone you want checked. It cannot change anything. We store it encrypted and you can delete it at any time.
Related guides
Cloudflare blocking Googlebot: how to find the setting and fix it
Why Cloudflare can return 403 to Googlebot — AI training blocks, Bot Fight Mode, WAF rules — and how to confirm and fix it without hurting your bot protection.
Guide · 6 minCloudflare AI Crawl Control: which setting to pick (and which one removes you from Google)
Search, Agent and Training in Cloudflare AI Crawl Control: what each controls, Block vs Disallow AI Training, the September 2026 changes and safe defaults.
Guide · 5 minDoes Cloudflare Bot Fight Mode hurt SEO?
What Bot Fight Mode challenges, why it is hard to whitelist on the Free plan, how it affects crawlers and monitoring, and when to use Super Bot Fight Mode instead.
More free tools
Googlebot checker
Free Googlebot test: fetch any URL as Googlebot Smartphone and Desktop, see robots.txt rules, noindex, CDN blocks and what content Google receives.
AI crawler checker
Check whether AI crawlers can access your website: GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-User, PerplexityBot. robots.txt and real CDN responses.
robots.txt tester
Test your robots.txt online: see which group applies to Googlebot, Bingbot, GPTBot, ClaudeBot and other crawlers, and which rule matches your URL.
Test from different countries
Check if your website loads from different locations and whether visitors or crawlers in some regions get blocked by a CDN, WAF or geo rule.