Where does ChatGPT get its information?

Short answer: ChatGPT's answers come from three separate sources, and almost every explanation of this conflates them. Training data is the frozen corpus the model learned from, current only to its cutoff. A retrieval index is a live search layer it queries when a question needs current information — this is where citations come from. On-demand fetching is what happens when you paste a URL and it goes and reads that page. The three run on different clocks, respond to different controls in your robots.txt, and only the second one decides whether your brand gets named when somebody asks for a recommendation.

Getting this wrong is expensive. Most companies that "block AI" block all three, which protects nothing they cared about and removes them from the answers they wanted to be in.

The three pipelines

	Training data	Retrieval index	On-demand fetch
What it is	The corpus the model learned from	A live index searched at answer time	A single page fetched right now
When it updates	Only at the next model release	Continuously	Per request
Produces citations?	No	Yes	Yes
The crawler	GPTBot	OAI-SearchBot	ChatGPT-User
Blocking it costs you	Presence in future models	Presence in answers today	Pasted-link lookups

The middle column is the one that matters for visibility, and it is the one people block by accident.

Training data: the frozen half

When ChatGPT answers a question that needs no current information — "what is a Wilson interval", "write me a regex" — it is drawing on weights learned during training. Nothing is looked up. There are no citations because there is no source to cite; the information has been compressed into parameters and cannot be traced back to a page.

OpenAI's own documentation describes the corpus as a mix of publicly available internet data, licensed third-party data, and data from users and human trainers. GPTBot is the crawler that collects the first category, and robots.txt is how you opt out of it.

Two things follow, and both are routinely misunderstood:

• Blocking GPTBot does not remove you from ChatGPT's answers. It affects what future models learn. Today's answers are unaffected.
• robots.txt is forward-looking. Content already crawled and already inside a training corpus is not withdrawn by a later Disallow. Blocking is a decision about future models, not a deletion request.

If your reason for blocking is content licensing — you do not want your writing training a commercial model — this is the pipeline to block, and blocking it is coherent. If your reason is anything to do with visibility, you are blocking the wrong thing.

The retrieval index: where citations actually come from

Ask ChatGPT which project management tool a small team should use, and it does not answer from memory. It runs a search, retrieves a set of sources, and synthesizes an answer over them — with links. That retrieval layer is built by a different crawler, OAI-SearchBot, and it is the pipeline that decides whether your brand appears in a recommendation.

This is the single most important distinction on this page. GPTBot and OAI-SearchBot are separate tokens with separate jobs, and a robots.txt that blocks both to "keep AI out" has quietly removed the company from every ChatGPT answer in its category while achieving a training opt-out it could have had on its own.

The same split exists elsewhere. Anthropic runs ClaudeBot for training and Claude-SearchBot for indexing. Perplexity runs PerplexityBot for its index and Perplexity-User for live fetches. Google is the confusing one: Google-Extended is not a crawler at all but a control token, and because a single Googlebot fetch feeds both Search and AI grounding, there is no clean way to be in one and not the other. We keep the full breakdown, with each operator's IP-verification source, in the AI crawler list.

On-demand fetching: the third one

When you paste a URL into ChatGPT and ask it to summarize the page, nothing about training or the index is involved. It fetches that URL, right then, as ChatGPT-User. Block that token and the feature simply fails for anyone who tries it on your site — including your own prospects, who do this more often than you would guess when they are evaluating a vendor.

This pipeline has no memory. It does not add you to an index and it does not improve your odds of being recommended tomorrow. It is a convenience feature, and it is worth leaving open for the same reason you leave a preview image working.

How to tell which pipeline produced an answer

There is a reliable tell: citations.

• An answer with linked sources came through retrieval or an on-demand fetch. It reflects the live web, and your presence in it is something you can work on.
• An answer with no sources came from training weights. It reflects the web as it was at the cutoff, and there is no lever you can pull this quarter that changes it.

This is also why "I asked ChatGPT and it said something wrong about us" needs a follow-up question before it becomes an action item. If the wrong claim came with a citation, there is a page to go fix or outrank. If it came from training, the honest answer is that it will persist until the next model, and the useful response is to make the correct information abundant and well-structured so the next corpus and the current index both carry it.

What this means if you want to be recommended

The practical order of operations, given the above:

1. Let the indexing crawlers in. OAI-SearchBot, PerplexityBot, Claude-SearchBot. This is necessary and not sufficient, and it is the cheapest thing on the list. You can check yours in about ten seconds with our free readiness checker.
2. Decide about training separately. Blocking GPTBot is a defensible licensing position. Just do not do it expecting a visibility outcome in either direction.
3. Be present where the retrieval layer already looks. In most B2B categories that means third-party listicles, comparison pages and community threads far more than your own homepage. The index surfaces what it can find and trust, and your own marketing site is one source among many.
4. Write the comparison pages you are missing. "X vs Y" and "alternatives to X" are among the most common shapes of buyer question, and the engine quotes whoever wrote that page. If that is not you, it is a competitor.

At EchoRanked we measure the outcome of all this directly — running the questions buyers actually ask across ChatGPT, Perplexity, Gemini, Copilot and Google AI Overviews on a schedule, and reporting how often each one names you with a confidence interval around the number. The band matters because the answers move: same question, same day, different result.

The part nobody can tell you

How the retrieval layer ranks and selects sources is not published, by any provider. Anyone presenting you with a definitive list of "ChatGPT ranking factors" has inferred them, and inference from a black box is a hypothesis rather than a finding.

What is established is the shape of the pipeline described above — which crawler does what, what each token controls, and what carries citations. That is enough to make good decisions with, and it is considerably more than most advice in this area is built on.

Frequently asked questions

Does ChatGPT use Google or Bing for search?

OpenAI has not committed publicly to a single fixed provider for the retrieval layer behind ChatGPT search, and the arrangement has changed over time. What is observable is that OAI-SearchBot crawls the web on OpenAI's behalf and that ChatGPT returns cited links, so treating it as its own index you need to be present in is the sound working assumption — rather than assuming your Google rank carries over.

Is ChatGPT's information up to date?

It depends which pipeline answered. A retrieval-backed answer reflects the live web and carries citations. An answer from training weights reflects the corpus at the model's cutoff and carries none. The same conversation can produce both, one message apart, which is why "is it current" has no single answer.

Can I stop ChatGPT from using my content?

You can opt out of training by disallowing GPTBot in robots.txt, going forward — content already collected is not withdrawn. Blocking the indexing crawler stops your pages being retrieved for answers, which also stops you being cited. There is no setting that lets you appear in answers while refusing to be read.

Why does ChatGPT say something wrong about my company?

Check whether the answer carried a citation. If it did, a specific page is producing the error and can be fixed or outranked. If it did not, the claim is coming from training data and will persist until a future model, so the fix is making the correct version abundant, structured and easy to retrieve now.

Where to go next

If you want to confirm the crawlers can actually reach you, start with how to check if ChatGPT can read your website. If you are about to make blocking decisions, read the complete AI crawler list first — the three-jobs distinction above is the whole reason that page exists. And once you start measuring whether any of this moved, why your AI visibility score is probably noise explains why one check is never enough.