An llms.txt file is a plain markdown file you place at the root of your site to give AI models and large language models a curated, machine-friendly map of your most important pages and docs. Instead of letting a crawler guess at your structure, you hand it a clean, prioritized index. Here is what it is and how to build one in minutes.
What an llms.txt file is
The format is simple markdown: an H1 title for your site, a blockquote summary that explains what the site is about, then lists of markdown links grouped under headings like Docs, Guides, or Products. AI tools that support the convention can read it to understand which pages matter most and what they contain.
It lives at yoursite.com/llms.txt, the same way robots.txt lives at the root. The goal is not to control access but to point models at your best content in a form they can parse quickly.
llms.txt vs robots.txt
These two files are easy to confuse, but they do opposite jobs:
- robots.txt tells crawlers what they may not access. It is a set of rules and exclusions.
- llms.txt invites models toward your best content. It is a curated map, not a gate.
One worthwhile caveat: llms.txt is an emerging convention, not an official standard. Adoption varies between AI providers, and there is no guarantee a given model reads it. Treat it as a low-cost, forward-looking signal rather than a requirement.
How to generate your llms.txt
You do not need to write the markdown by hand. The llms.txt Generator builds a valid file from your inputs:
- Open the llms.txt Generator and enter your site name and a one-line summary.
- Add your key pages as link entries, grouping them under headings such as Docs or Guides.
- Reorder so your most important pages sit near the top.
- Copy the generated markdown and save it as
llms.txt. - Upload it to your site root so it resolves at
yoursite.com/llms.txt.
Everything runs in your browser. Your URLs and page descriptions stay on your device and are never sent to a server.
What to include
Lead with the pages that best explain your site: documentation, getting-started guides, core product or service pages, and any reference material you want surfaced. Skip thin or duplicate pages. A short, well-curated file beats a long dump of every URL, because the whole point is to signal priority.
Related tools
- Block AI Crawlers (robots.txt): the flip side, for telling AI bots what to stay away from.
- LLM Token Counter & Cost Estimator: measure how much of your content a model has to read.
- System Prompt Generator: shape how an AI assistant responds once it has your context.
Write your llms.txt once, drop it at your root, and give every AI model a clean map of your best work.