llms.txt is a proposed standard: a Markdown file at your site's root that gives AI models a curated map of your most important content. Jeremy Howard of Answer.AI introduced it in September 2024 to help language models read documentation at inference time. As of 2026 no major AI provider confirms using it for search.

This guide covers what llms.txt is, how to write one in a few minutes, and the honest state of whether it does anything, so you can decide if it belongs on your list or near the bottom of it.

What is llms.txt?

llms.txt is a Markdown file, placed at yourdomain.com/llms.txt, that points AI language models to the content on your site worth reading. The idea is simple: model context windows are small, and a raw HTML page buried in navigation, ads, and scripts is hard to parse cleanly, so you hand the model a short, curated index instead. Jeremy Howard of Answer.AI published the proposal in September 2024 (llmstxt.org).

The format is deliberately plain. It has one required element, an H1 with your site or project name, then an optional one-line summary in a blockquote, optional free text, and H2 sections that list links to your key pages with a short note on each. It is aimed at inference, the moment a user asks an AI assistant about your product or docs, more than at model training.

Does llms.txt actually work for AI search?

Not yet, at least not for search visibility. Google's own generative-AI guidance states, in a section titled "mythbusting," that machine-readable files like llms.txt are not needed to appear in AI features (Google Search Central, 2026). John Mueller of Google's Search team put it more bluntly, calling llms.txt "not done for search" and comparing it to the old keywords meta tag: a claim a site makes about itself that engines can verify by reading the site directly (Search Engine Journal, 2026).

The usage data matches. Ahrefs analyzed 137,000 domains and found that of the roughly 38,000 with a valid llms.txt, 97% received zero requests for the file in May 2026, and AI retrieval bots like PerplexityBot and OAI-SearchBot accounted for only 1.1% of the requests that did occur (Ahrefs, 2026). If your goal is getting cited in ChatGPT or Google's AI answers, llms.txt is not the lever; the work in our answer engine optimization playbook is.

When is llms.txt worth adding?

llms.txt earns its place when you publish technical documentation that people load into AI coding assistants. That is the use case it was built for. Anthropic serves a live llms.txt for its API docs, documentation platforms like Mintlify generate the file automatically, and in Ahrefs' study Claude-Code was the second most active bot fetching these files, ahead of every AI search and assistant bot (Ahrefs, 2026). If you run developer docs, an API reference, or a large knowledge base, the file is genuinely useful to those readers today.

For a typical marketing site with no developer audience, the calculus is different. The file is quick to make and low risk, so adding one as a small hedge is reasonable. Just size the effort to the payoff: it is a ten-minute hedge with no proven search benefit, so keep your attention on the work that does move AI visibility. If you would rather hand the whole program to a team, that is the kind of work we take on at IG.

How do you create an llms.txt file?

You create an llms.txt file by writing a short Markdown document and hosting it at your root URL. Here is the process end to end.

  1. Create a plain text file named llms.txt.
  2. Add an H1 with your site or product name. This is the only required line.
  3. Add a one-sentence summary in a blockquote, then any short context that helps a model use the rest of the file.
  4. Group your best links under H2 headings, each as [name](url): short note. Link to clean, Markdown-friendly pages where you can.
  5. Put anything skippable under an H2 named ## Optional, so tools can drop it when context is tight.
  6. Host it at https://yourdomain.com/llms.txt and keep it current.

A minimal example:

# Innovative Group

> Innovative Group is an operating company with six specialty teams across strategy, technology, AI, and capital.

## Core pages
- [Answer Engine Optimization](https://innovativegroup.io/aeo/): How to get cited in AI answers.
- [Next Best Action](https://innovativegroup.io/next-best-action/): Measuring and improving AI citation share.

## Optional
- [Insights blog](https://innovativegroup.io/blog/): Weekly SEO, AEO, and GEO playbooks.

The spec also describes an optional /llms-full.txt that inlines the full content of your linked pages into one file, useful when a tool wants everything in a single pass (llmstxt.org). Generators exist if you would rather not write it by hand, though a short file you control is easy enough to maintain yourself.

How is llms.txt different from robots.txt and sitemap.xml?

The three files solve different problems, and llms.txt does not replace either of the others. robots.txt tells crawlers what they are allowed to access, and it is what OpenAI, Anthropic, and Google actually honor for AI crawling today. sitemap.xml lists every indexable page for search engines. llms.txt is a curated, human-and-model-readable summary of your best content for use at inference time (llmstxt.org). Keep robots.txt and sitemap.xml accurate first, since those carry real weight with search and AI crawlers today; our 2026 website audit checklist covers both.

What should you do instead for AI visibility?

Put your effort into the signals AI engines demonstrably use. Keep the site crawlable and fast, structure content so a model can lift a clean answer, and earn a presence across the web that models trust. That off-page work is the focus of our POV Extra Muros, and measuring your actual citation share is what IG's Next Best Action is built for. Add llms.txt if it fits your audience, then spend the rest of your time where the evidence points.