Definition
The fetcher an AI company uses to read the web: GPTBot and OAI-SearchBot (OpenAI), PerplexityBot, ClaudeBot (Anthropic), Google-Extended (Google's AI training opt-out control), CCBot (Common Crawl). Named in robots.txt to allow or block.
In the report
The heaviest of the thirteen checks: a site-wide Disallow for GPTBot, OAI-SearchBot, PerplexityBot, Google-Extended, ClaudeBot or CCBot fails it, and it carries six times the weight of a canonical tag, because a block undoes every other item.
Worked example
GPTBot fetches for ChatGPT's training, OAI-SearchBot for ChatGPT search, PerplexityBot for Perplexity, Google-Extended for Gemini. All eight sites in the September 2026 runs allowed them, which is why none of the eight had a crawling problem and all of them had a citation problem. A robots.txt with 'Disallow: /' for GPTBot but nothing for OAI-SearchBot still lets ChatGPT search read the site; a blanket disallow removes the site from every engine's answers. The report's readiness checks read the file and name which crawlers are blocked.
How to move it
Allow the search crawlers (OAI-SearchBot, PerplexityBot) and the fetchers (ChatGPT-User, Perplexity-User). Decide about the training ones (GPTBot, CCBot) separately. The crawler checker shows what your robots.txt says today.
Where it shows up
What is AEO (answer engine optimisation)? The work of being named and cited in the answers of engines like ChatGPT, Perplexity and Google's AI Overviews. It has two halves: making your own site readable and quotable (crawler access, structured data, plain statements of price and scope), and being present on the third-party pages the engines read before they answer.
Does blocking GPTBot hurt my visibility in ChatGPT? Not in ChatGPT search. GPTBot is OpenAI's training crawler; the search index is built by OAI-SearchBot, and live reads are done by ChatGPT-User. Blocking GPTBot keeps your pages out of future training data and leaves search alone. Blocking OAI-SearchBot or ChatGPT-User removes you from answers.
See AI crawler measured on your site
Your website, region and work email. The score appears in about three minutes, and the report is a link you keep.