# Lyrenth

> Lyrenth is the AI-readable web index: a search index of the public web built for AI. Any public URL becomes a stable AIDocument: clean markdown, source trace, cache truth, page structure, and token economics in one JSON contract. Built for agents. Origin-friendly by default.

Lyrenth is the AI-readable web index: a search index of the
public web built for AI. One standing index of 1.1B+ pages, each page
served as a stable AIDocument: clean markdown, source trace, cache
truth, page structure, and token economics in one JSON contract. The
same envelope for every page, for every agent. A query endpoint (send
a question, get the best-matching AIDocuments) is coming. Built for
agents. Origin-friendly by default.

This file follows the llms.txt convention (https://llmstxt.org/). If
you're an AI agent reading this, you can pivot from here to the
canonical pages below; or, if you want every URL on the web in this
same shape, read it through the index: POST https://api.lyrenth.com/v1/aidocument with
{"url":"..."}.

## Documentation

- [API quickstart](https://www.lyrenth.com/docs/quickstart): call POST /v1/aidocument from curl, Python, or Node.
- [AIDocument shape](https://www.lyrenth.com/docs/aidocument): the canonical v2 JSON envelope /v1/aidocument returns.
- [API documentation](https://www.lyrenth.com/docs): every endpoint, every parameter, every error code.
- [Pricing](https://www.lyrenth.com/pricing): Free, Starter Beta, and Pro Beta limits for AIDocuments, API rate, and verified domains.

## Blog

- [Blog index](https://www.lyrenth.com/blog)
- [Add web reading to Claude Desktop, Claude Code, or Cursor with one MCP block](https://www.lyrenth.com/blog/add-web-reading-to-claude-cursor-mcp): A five-minute quickstart: drop one MCP block into your assistant and it reads any URL as a clean AIDocument. Covers read_url, read_urls, and check_usage.
- [Agents don't browse, they read: what an index for machine readers looks like](https://www.lyrenth.com/blog/agents-dont-browse-they-read): Humans browse the web through layout and pixels; AI agents read it as text, structure, and provenance. That difference inverts how an index should be built.
- [How to feed web pages to an LLM without blowing the context window](https://www.lyrenth.com/blog/feed-web-pages-to-llm-without-blowing-context): Raw HTML burns your context window on nav, scripts, and boilerplate. Here is how to feed web pages to an LLM as clean AIDocuments, with real token numbers.
- [The publisher's guide to AI bots, robots.txt, and getting represented correctly](https://www.lyrenth.com/blog/publishers-guide-ai-bots-and-verification): Who reads your site now, what a well-behaved AI-index bot looks like, how robots.txt and llms.txt apply, and how to verify your domain with Lyrenth for free.
- [What is an AIDocument? One clean shape for agents that read the web](https://www.lyrenth.com/blog/what-is-an-aidocument): An AIDocument is one stable JSON shape for any web page: Markdown body, title, description, structure, and measured economics. Here is how it works.
- [Why your scraper gets empty HTML from JavaScript sites (and what rendering actually takes)](https://www.lyrenth.com/blog/why-scrapers-get-empty-html-from-javascript-sites): Fetch a React or SPA site and you get a hollow div, not content. Here is why JavaScript sites return empty HTML, and what real rendering takes at index scale.

## API endpoints

- [GET /v1/stats](https://api.lyrenth.com/v1/stats): live count of indexed documents (public).
- [POST /v1/aidocument](https://www.lyrenth.com/docs/quickstart): read any URL through the index as an AIDocument (Bearer auth); served from the standing index, crawled only on a miss.
- [GET /v1/read?url=...](https://www.lyrenth.com/docs/integrations): the same read as /v1/aidocument, but a GET that returns clean Markdown. Paste-and-go for agents and shell (Bearer auth).
- [GET /v1/document](https://www.lyrenth.com/docs/aidocument): index lookup only (Bearer auth).
- [POST /v1/submit](https://www.lyrenth.com/docs): queue a URL for background indexing.
- [GET /aidocument.schema.json](https://api.lyrenth.com/aidocument.schema.json): JSON Schema (draft-07) for the v2 envelope.

## Index data

- [Web index stats](https://www.lyrenth.com/web-index/stats): aggregate signal coverage across our corpus.

## For agents

- [Integrations: MCP, SDKs, reader](https://www.lyrenth.com/docs/integrations): read the web through Lyrenth from an MCP client (npx -y lyrenth-mcp), a GET /v1/read, or the Python (pip install lyrenth) and TypeScript (npm install lyrenth-sdk) SDKs, with LangChain, LlamaIndex, and the Vercel AI SDK adapters.
- [Agent manifest (JSON)](https://www.lyrenth.com/api/agent-manifest): brand, capabilities, endpoints, document shape.
- [AIDocument JSON Schema (draft-07)](https://api.lyrenth.com/aidocument.schema.json): canonical contract for the v2 response shape served by POST /v1/aidocument.
- [Bot identification](https://www.lyrenth.com/bot): we crawl with User-Agent "AIWebIndex/2.0". RFC 9309 robots.txt is enforced (if we cannot read your robots.txt, we do not crawl you); add a Disallow rule to opt out, or allowlist to be indexed. Machine-readable TDM / AI-training reservations (TDM Reservation Protocol, robots noai/notrain) exclude a domain from corpus licensing, honored worldwide.
- [Open Graph image](https://www.lyrenth.com/opengraph-image): 1200x630 PNG, brand identity.

## For site owners

- [Verify your domain](https://www.lyrenth.com/sites/add): AI agents already read your pages. Verify and you author the canonical AIDocument every agent gets when it reads you through us, see exactly what AI extracts from each page today, and push a change to make it live in seconds. Additive, not exclusive: the labs can still read you directly, and we never ask you to make Lyrenth your only path to AI. Free, no revenue share. See the [crawler policy](https://www.lyrenth.com/crawler-policy).

## Optional

- [Long-form (every page concatenated as markdown)](https://www.lyrenth.com/llms-full.txt)
- [Sitemap (XML)](https://www.lyrenth.com/sitemap.xml)
- [robots.txt](https://www.lyrenth.com/robots.txt)