llms.txt Ecosystemllmstxt.org

Machine-Readable Instructions for AI

llms.txt is a Markdown file at your site root. It helps AI agents understand your site's structure, purpose, and navigation.

What is llms.txt?

robots.txt tells search engines how to index your site. llms.txt tells AI agents how to understand and navigate it.

Markdown Format

Simple, human-readable, easy to maintain

Site Root Location

Always at /llms.txt for easy discovery

Context for Agents

What your site does, how to navigate it

Example llms.txt

A well-structured llms.txt file for a typical e-commerce site.

# Example Store

> E-commerce platform selling electronics and home goods.

## Main Sections

- [Products](/products) - Browse all products by category
- [Cart](/cart) - Shopping cart and checkout
- [Account](/account) - User account management
- [Support](/support) - Help center and contact

## API Documentation

- [REST API](/api/docs) - Product catalog, orders, user management
- [Webhooks](/api/webhooks) - Order status, inventory updates

## Important Pages

- [Shipping Policy](/shipping) - Delivery times and costs
- [Returns](/returns) - Return policy and process
- [Privacy](/privacy) - Data handling practices

## Site Structure

Products are organized by category (Electronics, Home, Garden).
Each product page has reviews, specifications, and related items.
Checkout requires account creation or guest checkout.

## Contact

- Support: [email protected]
- Phone: 1-800-EXAMPLE
- Hours: Mon-Fri 9am-5pm EST

Three Tools for llms.txt

Generate, validate, and compare llms.txt files using CBrowser CLI tools.

Generate llms.txt

Crawl your site and generate an llms.txt file from the discovered structure, navigation, and content.

# Generate llms.txt from site crawl

npx cbrowser llms-txt generate https://example.com

# Generate with custom depth

npx cbrowser llms-txt generate https://example.com --depth 3

# Output to file

npx cbrowser llms-txt generate https://example.com --output ./llms.txt

The generated file is a starting point. Review and customize it for your site.

Validation Rules

What the validator checks for in your llms.txt file.

Rule	Required
File exists at /llms.txt	Required
Starts with # heading	Required
Contains site description	Required
Lists main sections/pages	Recommended
Includes API documentation links	Recommended
Has contact/support information	Recommended
Specifies content types available	Recommended
No broken internal links	Required

CI/CD Integration

Add llms.txt validation to your deploy pipeline to keep it up to date.

# .github/workflows/llms-txt.yml

name: Validate llms.txt

on:
  push:
    paths:
      - 'public/llms.txt'
      - 'src/**'
  pull_request:
    paths:
      - 'public/llms.txt'
      - 'src/**'

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install CBrowser
        run: npm install -g cbrowser

      - name: Validate llms.txt format
        run: npx cbrowser llms-txt validate ./public/llms.txt --base-url ${{ vars.SITE_URL }}

      - name: Check for drift
        run: |
          npx cbrowser llms-txt diff ${{ vars.SITE_URL }} --format json > drift.json
          if [ $(jq '.additions + .removals' drift.json) -gt 10 ]; then
            echo "::warning::llms.txt may be out of date"
          fi

Ready to Add llms.txt?

Generate your first llms.txt file and add it to your site root.

Run an Audit First Learn More at llmstxt.org