ChatGPT is the famous one. It is not the only one.
Every check runs against all 24 crawlers below, from 18 operators. The list is read straight out of the engine, so this page cannot drift from what the tool actually does.
Blocking any one of these costs you a different audience
They do not share a pipeline
OpenAI, Google and Anthropic each run their own crawlers and honour their own robots tokens. Allowing GPTBot does nothing for Gemini. Allowing Google-Extended does nothing for Claude. There is no single switch, which is exactly why sites end up half-visible without knowing it.
Some read an index instead of the web
ChatGPT Search and Microsoft Copilot both draw on Bing's index. Gemini and Google AI Overviews draw on Google's. If you are missing from an index, every product built on top of it is missing you too — no crawler permission fixes that.
Fetching and training are different permissions
GPTBot trains; OAI-SearchBot indexes for citations; ChatGPT-User fetches live when somebody asks about you. Blocking training while staying citable is a legitimate and common choice — but only if you know which token does which.
India's assistants are not only the American ones
Meta AI inside WhatsApp, Copilot inside Windows, and Gemini inside Android and Chrome all reach Indian buyers who never open a chat website at all. Those surfaces are covered here too.
3 crawlers tracked · 3 probed live against your server
| robots.txt token | What it feeds | If it is blocked | How we test it |
|---|---|---|---|
| OAI-SearchBot | ChatGPT Search citations | Cites you — or does not | Live probe |
| ChatGPT-User | ChatGPT live browsing (user asks about you) | Cites you — or does not | Live probe |
| GPTBot | GPT model training corpus | Major corpus lost | Live probe |
2 crawlers tracked · 1 probed live against your server
| robots.txt token | What it feeds | If it is blocked | How we test it |
|---|---|---|---|
| Googlebot | Google index — feeds AI Overviews + Gemini grounding | Cites you — or does not | Live probe |
| Google-Extended | Gemini grounding + AI Overviews eligibility | Cites you — or does not | robots.txt only |
2 crawlers tracked · 2 probed live against your server
| robots.txt token | What it feeds | If it is blocked | How we test it |
|---|---|---|---|
| PerplexityBot | Perplexity index + citations | Cites you — or does not | Live probe |
| Perplexity-User | Perplexity live fetch on a user question | Cites you — or does not | Live probe |
1 crawler tracked · 1 probed live against your server
| robots.txt token | What it feeds | If it is blocked | How we test it |
|---|---|---|---|
| bingbot | Bing index — powers ChatGPT Search + Copilot | Cites you — or does not | Live probe |
3 crawlers tracked · 3 probed live against your server
| robots.txt token | What it feeds | If it is blocked | How we test it |
|---|---|---|---|
| Claude-SearchBot | Claude search results | Major corpus lost | Live probe |
| Claude-User | Claude live fetch on a user question | Major corpus lost | Live probe |
| ClaudeBot | Claude model training corpus | Major corpus lost | Live probe |
1 crawler tracked · 1 probed live against your server
| robots.txt token | What it feeds | If it is blocked | How we test it |
|---|---|---|---|
| Amazonbot | Alexa answers | Smaller surface lost | Live probe |
1 crawler tracked · 0 probed live against your server
| robots.txt token | What it feeds | If it is blocked | How we test it |
|---|---|---|---|
| Applebot-Extended | Apple Intelligence / Siri answers | Smaller surface lost | robots.txt only |
1 crawler tracked · 1 probed live against your server
| robots.txt token | What it feeds | If it is blocked | How we test it |
|---|---|---|---|
| CCBot | Common Crawl — the corpus almost every open model trains on | Smaller surface lost | Live probe |
1 crawler tracked · 1 probed live against your server
| robots.txt token | What it feeds | If it is blocked | How we test it |
|---|---|---|---|
| DuckAssistBot | DuckAssist answers | Smaller surface lost | Live probe |
1 crawler tracked · 1 probed live against your server
| robots.txt token | What it feeds | If it is blocked | How we test it |
|---|---|---|---|
| meta-externalagent | Meta AI / Llama training | Smaller surface lost | Live probe |
1 crawler tracked · 0 probed live against your server
| robots.txt token | What it feeds | If it is blocked | How we test it |
|---|---|---|---|
| AI2Bot | OLMo open training corpus | Minor | robots.txt only |
1 crawler tracked · 0 probed live against your server
| robots.txt token | What it feeds | If it is blocked | How we test it |
|---|---|---|---|
| Bytespider | Doubao / TikTok AI | Minor | robots.txt only |
1 crawler tracked · 0 probed live against your server
| robots.txt token | What it feeds | If it is blocked | How we test it |
|---|---|---|---|
| cohere-ai | Cohere training | Minor | robots.txt only |
1 crawler tracked · 0 probed live against your server
| robots.txt token | What it feeds | If it is blocked | How we test it |
|---|---|---|---|
| Diffbot | Knowledge-graph vendors | Minor | robots.txt only |
1 crawler tracked · 0 probed live against your server
| robots.txt token | What it feeds | If it is blocked | How we test it |
|---|---|---|---|
| PetalBot | Petal AI search | Minor | robots.txt only |
1 crawler tracked · 0 probed live against your server
| robots.txt token | What it feeds | If it is blocked | How we test it |
|---|---|---|---|
| MistralAI-User | Le Chat live fetch | Minor | robots.txt only |
1 crawler tracked · 0 probed live against your server
| robots.txt token | What it feeds | If it is blocked | How we test it |
|---|---|---|---|
| Timpibot | Timpi index | Minor | robots.txt only |
1 crawler tracked · 0 probed live against your server
| robots.txt token | What it feeds | If it is blocked | How we test it |
|---|---|---|---|
| YouBot | You.com answers | Minor | robots.txt only |
Two things this list does not claim
“robots.txt only” means exactly that
Some tokens are permission signals rather than crawlers with their own user-agent — Google-Extended and Applebot-Extended are honoured by another bot entirely. We read those from your robots.txt and do not pretend to have knocked on your door as them.
A live 200 is not a guarantee of citation
Being reachable is the prerequisite, not the result. Whether an engine actually names you is measured separately, by running buyer questions across 4 answer engines repeatedly — that is the paid tier, and we keep the two numbers apart on purpose.
Find out which of them can reach you.
All 24 are tested in a single free check, and the report shows the answer for each one separately.