Is your site blocking ChatGPT & AI search?
See which AI crawlers your robots.txt allows or blocks, from Google AI Overviews to ChatGPT, Perplexity and Claude, and get rules you can paste straight in.
Which AI crawlers should you allow?
Not every AI bot does the same job. Blocking the wrong one quietly removes you from AI answers.
AI search & assistant crawlers
Googlebot (AI Overviews), OAI-SearchBot and ChatGPT-User, PerplexityBot, Claude-SearchBot and Claude-User fetch your pages to answer questions and link to you as a source. Block one and that product can't cite you.
AI training crawlers
GPTBot, ClaudeBot, Google-Extended, CCBot and similar bots collect pages to train models. Blocking them is a legitimate choice and doesn't remove you from AI search results.
AI crawler questions
No. GPTBot only collects training data. ChatGPT search uses OAI-SearchBot, and ChatGPT-User fetches pages when someone asks it to. You can block GPTBot and still appear in ChatGPT answers, as long as those two are allowed.
Regular Googlebot. Google-Extended only controls whether your content is used for Gemini training and grounding; blocking it doesn't remove you from AI Overviews or AI Mode. To stay out of AI Overviews you'd have to use nosnippet, which also removes your normal search snippet.
The bot can crawl your site, but at least one path is disallowed for it, such as /wp-admin/ or /cart/. That's normal and usually fine.
Yes. Cloudflare's “Block AI bots” setting and bot-fight rules can stop crawlers at the firewall, which this tool can't see because it only reads robots.txt. Check Security → Bots in your Cloudflare dashboard as well.
Yes. It fetches your public robots.txt and applies the same rules the crawlers use. No signup, no API key.
Next: give AI assistants an llms.txt
Generate a ready-to-upload llms.txt from your sitemap, or run the full audit to check all 7 AI search visibility signals.
Rated 4.8/5 from 126 ratings