What is llms.txt, should you care about it?

Developers and marketers are told to add the llms.txt file to their website to help large language models (LLMSs) “know” their content.

But what exactly is LLMS.TXT and who is using it? More importantly, do you care?

LLMS.TXT is the recommended standard for helping LLMS access and interpret structured content on the website. You can read the full advice about it llmtext.org.

In short, this is a text file designed to tell llms nice one: API documentation, return policies, product taxonomy and other context resources. The purpose is to disambiguate by providing a curated graph of high-value content for the language model, so they don’t have to guess what is important.

Screenshot of the proposed standard at https://llmstxt.org/.

In theory, this sounds like a good idea. We’ve used something like robots.txt and sitemap.xml Help search engines understand what is on the website and where it is located. Why not apply the same logic to LLMS?

But what is important is, There are currently no major LLM providers that support LLMS.TXT. Not Openai. Not human. Not Google.

As I said in the introduction, llms.txt is Suggested standard. I could also come up with a standard (let’s call it “Please seed-I-Trade-Trail”, but it’s pointless unless the major LLM provider agrees to use it.

This is ours with llms.txt: This is a speculative idea that has not been formally adopted.

This is what the LLMS.TXT file looks like in practice. This is The actual llms.txt file of humans:

llms.txt is a core Price reduction Document (a text file in a special format). It uses the H2 header to organize the link to critical resources. Here is the sample structure you can use:

# llms.txt
## Docs
- /api.md
A summary of API methods, authentication, rate limits, and example requests.
- /quickstart.md
A setup guide to help developers start using the platform quickly.
## Policies
- /terms.md
Legal terms outlining service usage.
- /returns.md
Information about return eligibility and processing.
## Products
- /catalog.md
A structured index of product categories, SKUs, and metadata.
- /sizing-guide.md
A reference guide for product sizing across categories.

You can complete your own llms.txt in minutes:

1. Start with basics Tag files.
2. Use H2S to group resources by type.
3. Link to structured, price-reduced content.
4. Stay updated.
5. Host it in your root domain: https://yourdomain.com/llms.txt

You can create it yourself or use the free llms.txt Generator(Like this) serve you.

I’ve read that some developers have also experimented with LLM specific metadata in their llms.txt files, such as tagging budgets or preferred file formats (but there is no evidence that this is respected by crawling or LLM models).

You can see a list of companies using llms.txt Directory.llmstxt.Cloud– Community-maintained index of public LLMS.TXT files.

Here are a few examples:

• mintlify: Developer documentation platform.
• Bird: Real-time data API.
• Cloudflare: List performance and security documents.
• Human: Publish a full marker map of its API documentation.

But what about the big players?

So far, LLM major providers do not formally adopt LLMS.TXT As part of its crawling protocol:

• Openai (gptbot): Honor robots.txt, but not formally using llms.txt.
• Human (Claude): Releases its own LLMS.TXT, but does not state that its crawlers use the standard.
• Google (Gemini/Bard): Use Robots.txt (via user agent: Google-Extended) to manage AI crawling behavior without mentioning llms.txt support.
• Yuan (camel): There is no public crawler or guidance, nor is there any signs of llms.txt usage.

This highlights an important point: Creating an llms.txt is different from enforcing it in orbital behavior. Currently, most LLM vendors view llms.txt as an interesting idea rather than something they agree to prioritize and focus on.

I don’t think so, no.

There is no evidence that llms.txt can improve AI retrieval, improve traffic or improve model accuracy. And no provider promises to parse it.

But it’s also easy to set up. Compiling llms.txt is trivial if you already have structured content like product pages or developer documentation. This is a promotional document, hosted on your own website. There may be no observed benefits, but there are no risks. If LLM does ultimately follow the standards, early adopters may have some small advantages.

I think llms.txt is getting attention because we all want to influence LLM Visibilitybut we lack the tools to do so. So we grasp the idea Feel Like control.

But in my personal opinion, llms.txt is the solution to the problem. Search engines have crawled and understood your content using existing standards such as robots.txt and stitemap.xml. LLM uses many of the same infrastructure.

As Google’s John Mueller said Reddit Posts Recent:

AFAIK has no AI services and they say they are using llms.txt (you can tell you when to view server logs, or even not check them). To me, it rivals the keyword meta tags – this is what the site owner claims to be their website… (Is it really the site? Well, you can check it. Why not just check the site right then?)

Disagree with me, or want to share the opposite role model? Send me a message LinkedIn or x.