Is your site blocking ChatGPT, Claude or Perplexity?

Why check AI crawlers?

AI assistants now answer questions that used to be Google searches, and they cite websites as sources. If their crawlers can’t read your site, you can’t be recommended in ChatGPT search, Perplexity, Claude or Google’s AI features.

Not all AI bots are the same. Training crawlers (like GPTBot or CCBot) collect content to build models, and blocking them is a fair choice. Search and live-fetch bots (like OAI-SearchBot, ChatGPT-User or PerplexityBot) are what make your site show up in AI answers, so blocking them usually costs you visibility.

Many sites block AI bots by accident: a CDN or security plugin setting (for example Cloudflare’s “Block AI bots”) returns an error to them even though robots.txt allows them. That’s why we test each bot against the server too.