You've written a robots.txt. You've wrestled with sitemap.xml. Now there's a new three-letter file showing up in SEO threads everywhere: llms.txt. Some people call it the most important SEO addition of the decade. Others call it a gimmick. The truth sits somewhere in between, and it depends heavily on what kind of site you run.
What llms.txt actually is
llms.txt is a proposed convention: a plain Markdown file you place at your domain root (yoursite.com/llms.txt) that gives large language models a curated summary of your site. It lists who you are, what you offer, and links to your most important pages — docs, pricing, policies, APIs — in a format models can parse cheaply.
The idea was proposed by Jeremy Howard of Answer.AI in September 2024. The pitch is simple. Traditional crawlers index pages; robots.txt tells them which paths to avoid, sitemap.xml tells them what exists. But AI systems answering questions in ChatGPT, Claude, or Perplexity work under strict context limits — they can't read your entire site, your cookie banners, and three years of stale pop-ups. llms.txt hands them an answer key: here is what actually matters, and here is where to find it.
It's sometimes framed as the AI-era cousin of robots.txt. That's roughly accurate in shape — a small file at a well-known URL — but the audiences differ: robots.txt talks to crawlers about permissions; llms.txt talks to language models about priority content.
The awkward truth: what AI systems do with it today
Here is the part most hype pieces skip. As of 2026, none of the major AI search systems have said they consume llms.txt when generating answers — not OpenAI, not Google, not Anthropic, Meta, or Perplexity.
Google has been unusually direct. Gary Illyes confirmed at a Search Central Live event in July 2025 that Google does not support the format and has no plans to. John Mueller compared it to the keywords meta tag: a site describing itself is a claim the engine must still verify against the actual page, so it reads the page instead. Google's own AI optimization documentation, updated June 2026, states plainly that you don't need llms.txt for Google Search including its generative features, and that adding one won't harm or help visibility because Google Search ignores it. There's even a punchline: in December 2025 an llms.txt briefly appeared on Google's own developer docs site, generated by an internal system someone forgot to switch off. It vanished within hours and the URL now 404s.
Independent data tells the same story. An Ahrefs study of over 137,000 domains found that 97 percent of llms.txt files received zero traffic from AI crawlers. An analysis of more than 500 million bot traffic events found requests to llms.txt made up a negligible share of activity from the bots that feed AI search answers — GPTBot, ClaudeBot, PerplexityBot. A 300,000-domain study found no meaningful correlation between having the file and being cited in AI-generated answers; in fact, dropping llms.txt from the prediction model made the model more accurate. The file was adding noise, not signal.
Where it genuinely matters: developer docs and AI coding agents
So is llms.txt worthless? Not exactly. There is one corner of the ecosystem where it is genuinely useful: AI coding agents.
Tools like Cursor, Claude Code, Continue, and Cline increasingly load llms.txt when pulling in per-library documentation, and documentation platforms like Mintlify auto-generate llms.txt and a larger llms-full.txt for thousands of developer-docs sites. For a developer-tooling or API-docs site, publishing llms.txt is a clear net win: it helps agents quote your documentation accurately instead of guessing from third-party tutorials. Companies like Anthropic, Stripe, Cloudflare, and NVIDIA all publish one for their own docs — though none claim their crawlers consume third-party files.
For a regular business site or blog, the honest assessment is narrower: zero cost, and optionality if a major AI provider adopts the convention later. Generate one, but never present it as a Google ranking or citation lever.
How to write a good llms.txt (10 minutes)
If you've decided the ten minutes are worth it — and for most webmasters they are, if only defensively — here's how to do it properly.
1. Keep it Markdown and keep it small. The file is plain Markdown at /llms.txt. Start with an H1 naming your organization, one or two lines describing what it is, then curated sections. Models skim; keep the whole file tight rather than dumping your entire sitemap.
2. Structure it like a site map for models. A solid template:
# Acme Widgets
> One-line description of what the company does.
## Docs
- [Getting started](https://example.com/docs/start): first-run setup
- [API reference](https://example.com/docs/api): full endpoint docs
## Products
- [Pricing](https://example.com/pricing): plans and limits
## Policies
- [Privacy policy](https://example.com/privacy)Each link gets a short description so a model can decide whether the destination matters without fetching it.
3. Point at canonical, current pages. The single biggest failure mode is linking to stale content. If your llms.txt points a model at a 2019 pricing page while your real pricing changed in 2024, you've automated misinformation. Review the file whenever you ship site changes.
4. Don't confuse it with ai.txt. A different file, ai.txt (from Spawning AI), is about training-data consent — declaring whether sites may use your content to train models. llms.txt is a content map for inference-time systems. They're closer in spirit to a consent form versus a site map. Serving both is fine; treating them as interchangeable is wrong.
5. Validate and monitor. Serve it with a text/plain or text/markdown content type, check it loads at the root path, and glance at your server logs quarterly. Per the OtterlyAI server-log audit, only about 0.1 percent of AI-bot traffic hits /llms.txt — don't be alarmed if yours sits quietly.
Mistakes that quietly tank the whole exercise
- Treating it as an SEO ranking factor. It isn't one, at least not for Google. Prioritize page quality, crawlability, and structured data first — the things AI Overviews actually retrieve from.
- Letting agencies sell it as a citation booster. If a vendor promises llms.txt will get you cited in ChatGPT answers, the 300,000-domain study says otherwise. The honest pitch is future-proofing.
- Copy-pasting your homepage marketing copy. The file should be factual and navigational, not a sales brochure. Models benefit from precision, not adjectives.
- Forgetting maintenance. A stale llms.txt is worse than none: it confidently points agents at wrong answers.
The bottom line for webmasters
Add llms.txt as a cheap, ten-minute piece of future-proofing. Publish it if you run developer docs, an API, or any site where AI coding agents might pull your documentation. Don't add it expecting Google traffic or AI citations — those come from having crawlable, high-quality pages that engines already understand, same as ever.
Key takeaways
- llms.txt is a Markdown file at your domain root (proposed by Jeremy Howard in 2024) that gives AI models a curated map of your site's most important pages.
- Google ignores it: no support, no plans, and no ranking or AI-citation effect — its AI features retrieve from the regular search index instead.
- Studies of 137k–300k domains found virtually no correlation between having llms.txt and being cited by AI systems.
- Its real value today is with AI coding agents (Cursor, Claude Code, Mintlify docs sites), making it a clear win for developer-documentation sites.
- Write it in 10 minutes: Markdown, canonical links with descriptions, keep it current, and never present it as an SEO lever.