Making Your Website Agent-Friendly

AI agents are increasingly browsing the web — not with eyes, but with HTTP requests and accessibility trees. If your site isn't set up for them, they'll struggle to understand what you offer and how to interact with it. Here's how to fix that.

## The Problem

When a human visits your website, they see layout, color, hierarchy. When an agent visits, it sees raw HTML or an accessibility tree. Most websites are built exclusively for the first case. Agents end up guessing where to go, what endpoints exist, and how to authenticate.

## 1. Serve `llms.txt` at the Root

The llms.txt standard gives agents a concise, markdown-formatted site map. Think of it as robots.txt but for AI — it tells agents what your site does and where to find things.

request

GET /llms.txt

example llms.txt

# My Platform

> A short description of what this site does.

## Docs
- [API Reference](https://example.com/docs)
- [Getting Started](https://example.com/guide)

## Auth
- Method: API key via `Authorization` header
- Register: https://example.com/register

Keep it short. The agent reads this first to orient itself.

## 2. Serve `llms-full.txt` for Deep Context

Some agents want everything in one shot — full documentation, API details, examples. Serve a comprehensive markdown file at /llms-full.txt that combines your site map with complete documentation.

request

GET /llms-full.txt

This avoids the agent making dozens of follow-up requests to piece together your docs.

## 3. Add `/.well-known/agent.json`

Following the pattern of /.well-known/ discovery, serve a JSON manifest that describes your platform in a structured, machine-readable way.

{
  "name": "My Platform",
  "description": "What it does in one sentence.",
  "url": "https://example.com",
  "authentication": {
    "type": "api_key",
    "header": "Authorization",
    "registration_url": "https://example.com/register"
  },
  "capabilities": ["api", "file-storage", "webhooks"],
  "endpoints": {
    "docs": "https://example.com/docs",
    "api_base": "https://example.com/api/v1",
    "health": "https://example.com/health"
  }
}

This gives agents a structured entry point without parsing HTML.

## 4. Add Discovery Links in HTML `<head>`

For agents that start by fetching your homepage HTML, add <link> tags so they can discover your agent-friendly resources.

<link rel="alternate" type="text/markdown" href="/llms.txt" title="LLM Site Map">
<link rel="alternate" type="text/markdown" href="/SKILL.md" title="Full Documentation">
<link rel="author" type="application/json" href="/.well-known/agent.json" title="Agent Discovery">

## 5. Use Semantic HTML and ARIA Attributes

Agents that use browser automation (Playwright, Puppeteer) often rely on the accessibility tree rather than raw DOM. Semantic HTML makes this tree useful.

✓ do this

<main>
  <section aria-label="Registration">
    <form role="form"
      aria-label="Create account">
      <label for="email">Email</label>
      <input id="email" type="email"
        required
        aria-describedby="email-help">
      <span id="email-help">
        We'll send a verification code
      </span>
      <button type="submit">
        Register
      </button>
    </form>
  </section>
</main>

✗ not this

<div class="container">
  <div class="section">
    <div class="form-wrapper">
      <div class="input-group">
        <div class="label">
          Email
        </div>
        <input type="text">
      </div>
      <div class="btn"
        onclick="register()">
        Register
      </div>
    </div>
  </div>
</div>

The first example produces a clear accessibility tree. The second is a soup of anonymous div elements.

## 6. Add Structured Data (Schema.org JSON-LD)

Embed structured data in your HTML <head> so agents and search engines understand what your site is.

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "WebApplication",
  "name": "My Platform",
  "url": "https://example.com",
  "description": "What it does.",
  "applicationCategory": "DeveloperApplication",
  "potentialAction": {
    "@type": "RegisterAction",
    "target": "https://example.com/register"
  }
}
</script>

## Summary

What	Where	Purpose
`llms.txt`	/llms.txt	Concise markdown site map for agents
`llms-full.txt`	/llms-full.txt	Complete documentation in one file
`agent.json`	/.well-known/agent.json	Structured discovery manifest
`<link>` tags	HTML <head>	Agent discovery from homepage
Semantic HTML	All pages	Clean accessibility tree
JSON-LD	HTML <head>	Structured metadata

None of this breaks anything for human users. It's additive. Your site still works exactly the same — it just also works for agents.

Making Your Website Agent-Friendly

## The Problem

## 1. Serve llms.txt at the Root

## 2. Serve llms-full.txt for Deep Context

## 3. Add /.well-known/agent.json

## 4. Add Discovery Links in HTML <head>