What is llms.txt and does it actually work?

Short answer: llms.txt is a plain-markdown file at the root of your domain (/llms.txt) that hands language models a curated reading list of your most important pages. It is a proposed convention, not a standard anyone is obligated to honor. Writing one is cheap and low-risk, and it makes your site easier to summarize correctly — but as of mid-2026 no major AI provider has publicly confirmed that they fetch it at crawl time. So: useful as a discipline and a fallback, not a guaranteed ranking lever. Below is what the spec actually requires, a copy-paste annotated example, and a checklist to validate yours.

What the spec actually says

The llms.txt proposal was introduced in 2024 by Jeremy Howard of Answer.AI. It is deliberately tiny. A compliant file is markdown — not XML, not a new syntax — with a fixed shape:

A single H1 — your project or company name. This is the only strictly required element.
A blockquote summary — one paragraph of the key facts an unfamiliar reader needs to understand what you do.
Zero or more prose paragraphs/lists giving more detail (no headings).
H2-delimited link sections — each a list of [name](url): description entries pointing at the pages that matter.
An optional ## Optional section — links a model can safely skip when its context window is tight.

That's the whole specification. There are no crawl directives, no Disallow rules, no priority weights. It is a curated index, expressed as the kind of clean markdown a model parses well.

There's also a companion convention: serving a markdown version of any HTML page at the same URL plus .md (so /pricing also resolves at /pricing.md). That gives a model the clean text without the nav, cookie banners, and scripts.

An annotated example

Here is a complete, valid file. The comments () are illustrative — you'd delete them in production.

# EchoRanked
 
> EchoRanked runs real AI agents against your website, replays where they
> fail, and names the changes that fix the gaps — reported with confidence
> bands, not bare numbers.
 
EchoRanked is an AI-readiness tool for small teams. The free scan produces a
shareable scorecard; paid tiers add scheduled scans, agent buyer-task tests,
and the content agent.
 
## Docs                          <!-- H2 = a named link section -->
 
- [How it works](https://echoranked.com/how-it-works): The scan → fix → prove loop.
- [Methodology](https://echoranked.com/methodology): Sampling, confidence bands, and the api vs consumer measurement surface.
- [Pricing](https://echoranked.com/pricing): Tiers and what each scan includes.
 
## Blog
 
- [What is llms.txt](https://echoranked.com/blog/what-is-llms-txt): This guide.
- [JSON-LD for AI search](https://echoranked.com/blog/json-ld-for-ai-search): Which schema actually matters.
 
## Optional                      <!-- safe to skip under tight context -->
 
- [Changelog](https://echoranked.com/changelog): Release notes.

Three things make this file good rather than merely valid:

Absolute URLs. A model that reads /llms.txt out of context can't resolve relative links. Always write the full https:// URL.
Descriptions that disambiguate. "Pricing: tiers and what each scan includes" tells the model when to pick this link. "Pricing" alone does not.
It points at answer-shaped pages. A link to a page that buries its answer under three screens of brand copy wastes the model's attention. If a page can't be summarized in a sentence, fix the page before you list it — see how to check if ChatGPT can read your website.

Diagram — the two reading paths: Show a model encountering a domain by two routes. Path A (no llms.txt): the crawler hits raw HTML, fights through nav/scripts/ads, and guesses which pages matter. Path B (with llms.txt): the crawler reads a short curated index, follows named links to clean .md versions, and arrives at the answer in one hop. The diagram should make clear the file shortens the path to the right page — it does not force the model down it.

What it does not do

Being explicit here saves disappointment:

It does not control crawling. That is robots.txt. A bot blocked there never sees your llms.txt at all.
It does not guarantee citation. Models cite what their retrieval layer surfaces. llms.txt improves the odds that the right page is what gets surfaced; it can't manufacture demand for your content.
It does not replace structured data. JSON-LD still does the heavy lifting for prices, FAQs, and organization facts inside the page itself.

Does it actually work? An honest read

This is the question vendors gloss over, so here's the straight version.

No major AI provider — OpenAI, Anthropic, Google, or Perplexity — has publicly documented llms.txt as an input their crawlers fetch and use. Google's Search Advocate John Mueller has publicly downplayed it, comparing its likely fate to the long-ignored keywords meta tag. Server-log analyses circulating in 2025–2026 show inconsistent fetching of /llms.txt by AI user agents. So if anyone promises you a citation lift because you added the file, treat that as marketing, not measurement.

What the file genuinely buys you:

A forcing function. Writing a good llms.txt makes you decide which ten pages actually answer buyer questions. That clarity helps every channel.
A clean fallback surface. Where a tool or agent does read it (some developer-facing tools and indexers now do), you've handed it accurate, current facts instead of letting it guess.
Near-zero downside. It's a static text file. The only real risk is letting it go stale — see below.

The verdict: add it, keep it honest, automate its upkeep, and don't expect it to move a needle on its own. The page-level work — rendering, structured data, answer-first copy — is what actually changes whether models can read and quote you.

The failure mode we see most

Teams generate an llms.txt once, link forty pages, and never touch it again. Six months later half the links 404 and the blog section omits the posts that answer real buyer questions. A stale map is worse than no map: it teaches anything that does read the file that your site can't be trusted.

The fix is boring and durable: generate the file from the same source of truth as your sitemap, and fail your build when it drifts. That's how EchoRanked's own llms.txt is produced — and a stale-or-missing file is one of the first things our readiness scan flags.

Validator checklist

Before you ship /llms.txt, confirm:

Frequently asked questions

Is llms.txt the same as robots.txt?

No. robots.txt controls access — which crawlers may fetch which paths. llms.txt assumes access is already allowed and offers guidance — a curated list of the pages worth reading. They solve different problems, and a model blocked by robots.txt will never reach your llms.txt.

Where does the file go, and what should it be named?

It must live at the domain root and be named exactly llms.txt (lowercase), served over HTTPS at https://yourdomain.com/llms.txt. Subdirectory copies are ignored. Optionally, also serve markdown versions of key pages at their URL plus .md.

Will adding llms.txt improve my AI search rankings?

There's no evidence it directly improves citations, and no major provider has confirmed using it at crawl time. Treat it as a low-cost clarity exercise and a fallback for tools that do read it — not a ranking tactic. The work that measurably affects whether models can read you is page-level: crawler access, server-side rendering, and structured data.

Where to go next

If you want to know whether assistants can even reach your pages in the first place, start with how to check if ChatGPT can read your website. And once you've curated the map, make the pages themselves machine-legible with the JSON-LD that actually matters for AI search.