ChatGPT with web search runs one or more searches for the question, reads the top results, and writes an answer that names companies from those pages. Allow OAI-SearchBot and ChatGPT-User in robots.txt. State the facts a buyer asks for on the page: the price, the countries served, the integrations.
How ChatGPT decides who to name
ChatGPT with web search runs one or more searches for the question, reads the top results, and writes an answer that names companies from those pages. It leans on the same kind of pages a person would open: roundups, review sites, comparison articles, and a company's own site when the site states the thing plainly. It does not keep a list of companies per category; the list is assembled from the pages it fetched for that question, which is why two askings can differ.
What ChatGPT reads
| Agent | What it does |
|---|---|
OAI-SearchBot | builds the index ChatGPT search draws on; the one to allow if you want to be findable |
ChatGPT-User | fetches a page when a user's prompt sends ChatGPT to it; a live read, not an index |
GPTBot | collects pages for training future models; blocking it does not remove you from search |
The lines to have in robots.txt, checked in seconds by the crawler access checker:
User-agent: OAI-SearchBot
Allow: /
User-agent: ChatGPT-User
Allow: /How ChatGPT shows its sources
Links in the answer, attached to the sentences they support, and a sources list. The report reads both. A company can be named without being cited (the engine learned it from a roundup) and cited without being named (its page was read for a fact about someone else).
What the report records
The report uses OpenAI's Responses API with the hosted web search tool, the same search ChatGPT uses, asked with the buyer's country set. For each question it records the answer, the order of the names, and the cited URLs.
When ChatGPT is not scored in a report the header says why: our key was refused, our quota ran out, or it timed out. Those are faults on our side and the report is re-run.
What to do for ChatGPT
- Allow OAI-SearchBot and ChatGPT-User in robots.txt. A site-wide block on them is the single most expensive line a site can carry.
- State the facts a buyer asks for on the page: the price, the countries served, the integrations. ChatGPT quotes the page that states them and skips the one that gestures.
- Be on the roundups it reads. For most trades that is a review site and two or three comparison articles; the report names which ones were cited for your questions.
Read next
Perplexity · Gemini · Google AI Overviews · All four engines · By industry · Grounded answer, defined
Questions
Does blocking GPTBot hide me from ChatGPT search?
No. GPTBot is the training crawler. OAI-SearchBot is the search crawler, and ChatGPT-User the live fetcher. Block GPTBot if you want, and allow the other two.
Why does ChatGPT give a different list each time?
The list is assembled from the pages fetched for that asking, and the searches vary. The report asks each question once per engine and treats the answer as a sample; the plans rescan monthly on the same questions so the trend is visible through the noise.
See whether ChatGPT names you
Your website, region and work email. The score appears in about three minutes, and the report is a link you keep.