How to read this list
These are the AI crawler user agents amisearchable.cc checks in robots.txt. Search and fetch bots decide whether a site can be retrieved for live answers. Training crawlers are informational. The table is generated from the same list the checker uses, so it cannot drift from the product.
Search and fetch bots
| User-agent | What it does |
|---|---|
| OAI-SearchBot | Builds the ChatGPT search index. Blocking this hides you from ChatGPT search results. |
| ChatGPT-User | Fetches a page when someone asks ChatGPT about it. This is the live-answer bot, not the training crawler. |
| PerplexityBot | Crawls for Perplexity search. Separate from user-initiated fetches inside Perplexity answers. |
| Claude-SearchBot | Indexes the web for Claude search / citations. |
| Claude-User | Fetches a page when Claude needs the live contents to answer. |
Training crawlers
Blocking these does not fail the AI-Search Ready badge. Many sites block training and still want to appear in live answers.
| User-agent | What it does |
|---|---|
| GPTBot | Collects data to train OpenAI models. Blocking it does not by itself hide you from ChatGPT answers. |
| ClaudeBot | Collects data to train Anthropic models. Informational on the badge; blocking it is a common choice. |
| Google-Extended | Controls Gemini / Google AI training use. It is not Googlebot and does not control Google Search ranking. |
| CCBot | Common Crawl. Many models train on Common Crawl snapshots. Not a live answer bot. |
Not on this list, on purpose
Googlebot is a search-engine crawler, not an AI training token. Amazonbot and Bytespider exist, but they are outside the pass/fail rule this product uses today. If a vendor publishes a new search-fetch user agent, this page updates when the checker does.