Home / Free tools

Googlebot checker: can Google crawl your page?

Fetch a page the way Googlebot does — smartphone and desktop — and see exactly what stands between Google and your content: robots.txt, noindex, a CDN or WAF block, or a stripped-down page.

Free · No sign-up · About 10 seconds

What this check covers

Googlebot Smartphone & Desktop

Both user-agents, because Google indexes mobile-first but still crawls with desktop.

robots.txt per crawler

Which group applies to Googlebot and the exact rule that matches your URL.

noindex & X-Robots-Tag

Meta robots, googlebot-specific meta and response headers.

CDN / WAF response

Cloudflare, Fastly, Akamai, Imperva, Sucuri, DataDome — challenge pages detected even behind HTTP 200.

Content parity

Compares what Googlebot receives with what a visitor sees.

Sitemap reachability

Fetches the sitemap from robots.txt as Googlebot.

Questions

Is this the same as a real Googlebot visit?

No tool outside Google can send real Googlebot traffic. We send Googlebot's exact user-agent from our servers. Sites that verify Googlebot by IP will treat us as an impostor — we detect that pattern and tell you when a block is most likely just fake-bot protection. To confirm, use URL Inspection in Search Console or connect your Cloudflare account.

What replaced "Fetch as Google"?

Google replaced Fetch as Google with the URL Inspection tool in Search Console. It only works for sites you have verified. This checker works for any URL and adds CDN and AI-crawler checks that URL Inspection does not show.

Why does Googlebot get a 403 when my browser does not?

Usually a bot-protection rule: Cloudflare Bot Fight Mode, an AI-crawler block that also covers Googlebot, a WAF rule on user-agents, or a security plugin. See our guide on Cloudflare blocking Googlebot.

Related guides

Guide · 7 min

Cloudflare blocking Googlebot: how to find the setting and fix it

Why Cloudflare can return 403 to Googlebot — AI training blocks, Bot Fight Mode, WAF rules — and how to confirm and fix it without hurting your bot protection.

Guide · 6 min

“Blocked due to access forbidden (403)” in Google Search Console: causes and fixes

What the “Blocked due to access forbidden (403)” status means, the most common causes (CDN, WAF, security plugins, login walls) and a step-by-step fix.

Guide · 5 min

How to verify Googlebot (and spot fake Googlebots)

Verify that a request claiming to be Googlebot really comes from Google using reverse DNS or Google’s published IP ranges. Includes commands and common mistakes.