AI
llms.txt
What llms.txt is, why publishers are adopting it, and how it differs from robots.txt and the sitemap.
llms.txt is an emerging convention that gives LLM crawlers a structured, human-readable map of a site's most important content. Proposed by Jeremy Howard (Answer.AI) in late 2024, it sits at the root of a website alongside robots.txt and sitemap.xml. The format is Markdown with an H1 site name, a blockquote description, and H2 sections of bulleted links.
Adoption accelerated through 2025 as AI search engines (Perplexity, ChatGPT Search, Claude's Web Search, Google AI Overviews) signalled they were reading the file. Notifire was one of the early publishers to ship one; this hub tracks the spec's evolution and the publishers and tools that adopt it.