Definition
The fetcher an AI company uses to read the web: GPTBot and OAI-SearchBot (OpenAI), PerplexityBot, ClaudeBot (Anthropic), Google-Extended (Google's AI training opt-out control), CCBot (Common Crawl). Named in robots.txt to allow or block.
In the report
The heaviest of the thirteen checks: a site-wide Disallow for GPTBot, OAI-SearchBot, PerplexityBot, Google-Extended, ClaudeBot or CCBot fails it, and it carries six times the weight of a canonical tag, because a block undoes every other item.
How to move it
Allow the search crawlers (OAI-SearchBot, PerplexityBot) and the fetchers (ChatGPT-User, Perplexity-User). Decide about the training ones (GPTBot, CCBot) separately. The crawler checker shows what your robots.txt says today.
See AI crawler measured on your site
Your website, region and work email. The score appears in about three minutes, and the report is a link you keep.