Do ChatGPT, Perplexity, or Google actually read llms.txt?

No engine has confirmed consuming it as of mid-2026. Google's John Mueller publicly compared it to the keywords meta tag. It is a low-cost bet: adoption by tooling vendors is real, engine consumption remains speculative.

Is llms.txt the same as robots.txt?

No — they are opposites. robots.txt tells crawlers what they may not fetch; llms.txt tells LLMs what they should read first. One is an exclusion protocol, the other a curation layer.

Where does llms.txt go?

At the site root: https://example.com/llms.txt. The expanded llms-full.txt variant, containing full page content rather than links, sits alongside it.

What Is llms.txt? The Proposed Standard for LLM-Friendly Content

Last updated: July 5, 2026 · Menra Team

llms.txt is a proposed standard — a markdown file served at a site's root — that gives large language models a curated, token-efficient map of a site's most important content. Jeremy Howard of Answer.AI proposed it in September 2024, with the specification published at llmstxt.org.

Why was it proposed?

Context windows are finite and HTML is noisy: navigation, scripts, and boilerplate can consume ten times the tokens of the actual content. The proposal's premise is that an LLM fetching your site at inference time does better with a clean markdown index than with raw HTML. It targets the retrieval moment, not training — a distinction that separates it from robots.txt directives aimed at crawl permission.

What does the format look like?

The spec is deliberately simple markdown, in a fixed order:

An H1 with the site or project name (the only required element)
A blockquote summarizing what the site is
H2-labeled sections containing link lists, each link with a short description
An optional section named Optional for content that can be skipped when context is tight

A companion convention serves markdown versions of pages at the same URL with .md appended, so an agent can go from index to clean content without an HTML parse.

What is the adoption evidence?

Tooling adoption is real: Mintlify added automatic llms.txt generation for its documentation platform in November 2024, putting thousands of developer-docs sites on the standard at once, and companies including Anthropic and Cloudflare publish their own files. Engine adoption is the missing half — no major answer engine has confirmed reading the file, and Google's John Mueller likened it to the long-dead keywords meta tag. Directories like llmstxt.site track published files in the wild.

Should you implement it?

The cost is an hour of curation for a static site, near zero if your framework generates it. The upside is speculative but the downside is nil, which is why most GEO programs ship it as cheap insurance. Generate it from your sitemap, keep it under a few hundred lines, prioritize your highest-value evergreen pages, and regenerate it on deploy so it never drifts out of date. Treat it as a supplement to — never a replacement for — server-rendered HTML and structured data that engines demonstrably consume today.

Frequently asked questions

Do ChatGPT, Perplexity, or Google actually read llms.txt?: No engine has confirmed consuming it as of mid-2026. Google's John Mueller publicly compared it to the keywords meta tag. It is a low-cost bet: adoption by tooling vendors is real, engine consumption remains speculative.
Is llms.txt the same as robots.txt?: No — they are opposites. robots.txt tells crawlers what they may not fetch; llms.txt tells LLMs what they should read first. One is an exclusion protocol, the other a curation layer.
Where does llms.txt go?: At the site root: https://example.com/llms.txt. The expanded llms-full.txt variant, containing full page content rather than links, sits alongside it.

Keep exploring

See how AI engines talk about your brand — track mentions across ChatGPT, Perplexity, Claude, Gemini and 5 more. Start with Menra