Article
What is llms.txt and how to create it for my site?
The new file that instructs AI systems about what your site contains — and how to implement it for any industry
llms.txt is a simple text file placed at the root of the site (at yourdomain.com/llms.txt) that describes the content and structure of the site to artificial intelligence systems that access it directly — such as AI agents, language models with real-time search, and AI indexers. The proposal was created by Jeremy Howard (fast.ai) in 2024 and is not yet a universal standard, but its adoption is growing rapidly as a signal of citability intent.
What llms.txt is for
The problem llms.txt solves: when an AI system directly accesses a site to understand its content, it has no standardized way of knowing what the site is, what it offers, which pages are most important, and how the content is organized. llms.txt provides this information in machine-readable format, without requiring HTML processing or page navigation.
It's useful to think of llms.txt as a briefing you give the AI before it reads the site. Instead of letting the model infer from scratch what a law firm specializing in labor law does, llms.txt declares this directly — along with the most relevant pages, the service structure, and the types of content available.
Who should create an llms.txt
Any site that wants to maximize its citability in AI systems can benefit. Some examples of how different types of businesses use the file:
Fashion e-commerce: describes product categories (women's, men's, children's, accessories), brands carried, exchange policy, and fashion editorial pages that can be cited as trend references.
Medical clinic: describes specialties offered, accepted insurance plans, location, medical staff, and information pages about procedures and health conditions.
Law firm: lists practice areas (labor, tax, contracts), partner profiles, jurisdictions served, and legal blog articles.
Online course platform: describes learning tracks, instructors, formats (live, recorded, certification), segments served, and testimonial pages.
B2B software company: describes product modules, target segments, use cases, technical documentation, and pricing pages.
How to create an llms.txt
The format is simple Markdown text. The recommended structure:
``` # Site or company name
> Concise description of what the site is and who it serves (1 to 3 lines)
Site sections or areas
- Page or section name: brief content description
About the company
Paragraph with institutional information relevant for context.
Most relevant content for AI
- Article or page 1: description - Article or page 2: description ```
A simplified example for an accounting firm:
``` # Express Accounting — Accounting Firm for SMEs
> Accounting firm specializing in small and medium businesses. > We serve clients nationwide with focus on company formation, > monthly accounting, and tax planning.
Services
- Company Formation: how to register a business, tax regime selection, required documentation
- Monthly Accounting: bookkeeping, payroll, and tax obligations services
- Tax Planning: legal strategies for tax burden reduction
Blog
- LLC or S-Corp: which to choose?: tax regime selection guide ```
How llms.txt differs from robots.txt
| robots.txt | llms.txt | |
|---|---|---|
| Purpose | Controls which bots can crawl which pages | Describes site content to AI systems |
| Type of instruction | Permission (allow/disallow) | Description (what it is, what it has, where it is) |
| Adoption | Universal standard, decades of use | Emerging, growing adoption in 2024–2026 |
| Effect if ignored | Bot may ignore or block crawling | AI has no advance contextual description |
The two files are complementary — robots.txt defines access, llms.txt defines context.
Is it worth implementing now?
Yes, given the low implementation cost and the positive signal it sends. Creating an llms.txt takes less than an hour for most sites and requires no technical development — it's a text file created manually and published to the server root.
The risk of not implementing is minimal in the short term (engines still function without it), but the trend is toward growing adoption. Companies that implement early build an advantage as AI systems increasingly use the file.
FRT Digital includes llms.txt creation as part of the technical implementation of the AIO service. For a complete diagnosis of your site's technical gaps, start with the AIO Score audit.