llms.txt: what it is, who reads it, and whether your site needs one

The short answer

The honest verdict is in Google's own guide to generative AI features. You do not need to create special AI text files to appear in Google Search, and keeping one for other services "will neither harm nor help" your rankings (Google Search Central, updated 10 July 2026). So add one for the agents that may use it, spend ten minutes, and put the real effort elsewhere. This article explains where.

What is llms.txt?

llms.txt is a proposal "to help agents use a website": a short Markdown file that gives an AI tool background on a site and links to more detailed pages. Only the H1 title is required. A summary, notes and lists of links follow. Agents read the small file first and follow links only when they need more detail.

The proposal comes from Jeremy Howard, published on 3 September 2024 and revised as version 2 on 10 August 2026 (llmstxt.org). It describes itself as open for community input. It is not an IETF or W3C standard, and nothing obliges any crawler to read it.

The proposal also suggests offering a clean Markdown copy of each page, linked from the HTML with rel="alternate" type="text/markdown". That part is arguably more useful than the index file, because it gives an agent the page's words without the navigation and scripts.

Who actually reads llms.txt?

Fewer readers than the hype suggests. Google Search ignores it. Chrome's Lighthouse checks it. Developer documentation sites publish it, because coding assistants read documentation that way. No major answer engine, including ChatGPT, Claude, Perplexity or Google's AI features, has documented using llms.txt to decide what it cites.

Here is what we could verify on 11 October 2026.

• Google Search does not use it. Its guide says Search does not use these files (Google).
• Lighthouse checks for it. The Agentic Browsing category, in documentation updated in May 2026, includes an llms.txt audit. A server error fails it; a 404 is marked not applicable. Lighthouse calls the file "optional at the moment" and says without it agents may spend longer crawling to understand a site (Chrome for Developers).
• Shopify publishes one for every store. Its managed agents.md file is mirrored at /llms.txt by default, as our article on Shopify agentic commerce shows from real stores.
• The AI companies publish it for their own docs. Anthropic's developer documentation and OpenAI's developer site both serve an llms.txt today. That tells you where the format earns its keep: large documentation sets read by coding tools.

Does llms.txt help SEO or AI citations?

No, not on any evidence we can find. Google says it is neutral for Search rankings. No answer engine has published that it reads llms.txt when choosing sources. Treat it as a courtesy to agents that look for it, not as optimisation. Anyone selling llms.txt as a way to "rank in ChatGPT" is selling a guess.

What does decide AI visibility is unglamorous. Google's guide says a page must be indexed and eligible to show with a snippet to appear in its AI features. Our guide to getting cited by ChatGPT covers what OpenAI documents, and generative engine optimisation covers the wider picture.

A worked example: PostCliq's own llms.txt

We publish one, so here is what it looks like from the inside. postcliq.com/llms.txt is 128 lines and about 27 KB. It has an H1, a one-paragraph summary, and 105 links across six sections: product and company, articles, topics, comparisons, free resources, and an Optional section holding the sitemap and our crawler page.

Three choices are worth copying. First, the summary opens with a one-sentence definition of the business and then states its limits, such as money being shown only as an estimated range, so an agent has less room to overstate what we do. Second, the first link points to our official information page for AI systems, which states plainly what PostCliq does and does not do. Third, that page offers a Markdown version, linked as an alternate format, as the proposal suggests.

One honest criticism of our own file. At 27 KB it is longer than "concise" implies, because it lists every article. That suits an agent wanting a full map. A small business with twenty pages should aim for a short file: who you are, what you sell, where you work, and links to services, pricing and contact.

PostCliq is built by WP Creative, a WordPress agency in Sydney, and on WordPress the file is a settings switch. Yoast SEO's free plugin generates llms.txt, with an option to pick the pages yourself (Yoast). Whatever generates it, read the result and check it leads with services, pricing and contact rather than last month's blog.

What do AI agents actually need instead?

Our view: an agent visiting your site is a buyer's assistant on an errand. It needs to get in, read the facts, and find a way to act. llms.txt helps with the second step at the margin. Access, readable HTML, clear facts and a usable enquiry path decide whether the errand ends with your business on the shortlist.

We think this is where the web is heading. Assistants that browse websites and fill in forms for people are now sold to the public, such as Claude in Chrome. Personal assistants such as OpenAI's Dots and Meta's Muse now do the same in the background on their own cloud computers, as our guide to AI personal assistants and your website explains. Lighthouse already audits pages for agent accessibility, layout stability and WebMCP, alongside llms.txt. The site that gets the enquiry will be the one an agent can read and act on, not the one with the cleverest file.

Our agent readiness check scores all of this out of 100, reading your homepage, robots.txt, llms.txt and sitemap. Firewall access, robots.txt rules and server-rendered content carry the most points, because without them nothing else is read.

What to do next

1. Check access first. Make sure the search and user bots are allowed, as set out in our guide to ClaudeBot and Anthropic's other bots.
2. Run the agent readiness check to see what an agent can read on your site today.
3. Write a short llms.txt: an H1, a one-sentence definition of the business, and links to services, pricing, case studies and contact.
4. Put the same definition on your homepage, About page and structured data, so every source agrees.
5. Test your enquiry form without JavaScript, then run a free PostCliq audit for the rest of the enquiry path.

Frequently asked questions

Do I need an llms.txt file?

You do not need one to rank in Google or appear in AI answers. Google says it neither helps nor harms Search visibility. It is still worth ten minutes, because agents and tools that look for it get a quick, accurate summary of your business, and Lighthouse now checks for it. Fix crawl access and readable pages first.

Does llms.txt help SEO?

No. Google's guide to generative AI features says Search does not use llms.txt and that keeping one will neither harm nor help your rankings. No major answer engine has documented using it to pick citations either. Its value is helping agents orient quickly, not moving rankings.

Where does the llms.txt file go?

At the root of your domain, so it loads at yoursite.com/llms.txt, in the same place as robots.txt. Serve it as plain text or Markdown with a 200 response. Lighthouse fails the check only if the server errors when fetching the file; a missing file is marked not applicable.

What is the difference between llms.txt and robots.txt?

robots.txt tells crawlers what they may and may not fetch, and Anthropic and OpenAI both document it as the way to control their bots. llms.txt tells AI tools what your site is and which pages matter, and nothing obliges them to read it. One is a gate. The other is a signpost, and a signpost is useless if the gate is shut.