> ## Documentation Index
> Fetch the complete documentation index at: https://docs.qwairy.co/llms.txt
> Use this file to discover all available pages before exploring further.

# Site Readiness

> Check whether AI crawlers are allowed to read your site, and generate the files that guide them to your best content.

# Site Readiness

Before an AI engine can cite you, its crawler has to be allowed to read your site. Site Readiness is the access layer of GEO: it tells you **who is allowed in**, and gives you the files that tell AI systems what to read once they are.

Navigate to **Optimize > Site Readiness** in the left navigation menu.

## AI Crawler Readiness

Site Readiness checks your `robots.txt` against **63 known AI crawlers** — from the engines behind ChatGPT, Claude, Gemini, Perplexity, Copilot, and Mistral, down to the long tail of smaller bots.

For each crawler, the view shows:

* **Allowed or blocked** — the verdict for that specific bot against your current `robots.txt`.
* **The exact rule that applies** — so you know precisely which `Disallow`/`Allow` line is responsible, instead of guessing.
* **What the crawler is for** — a short description of the product or engine behind it.
* **Priority tier** — crawlers are grouped so the engines that matter most for your visibility are front and center, rather than buried in a long alphabetical list.

<Tip>
  Each crawler entry also shows a **last seen** time, pulled from your connected server logs. Readiness tells you who is *allowed* in; last seen tells you who actually *came*. Cross-reference with [Crawler Analytics](/documentation/measure/crawler-analytics) to see the full picture — a crawler can be allowed but never show up, which usually points to a discovery problem rather than an access problem.
</Tip>

## `robots.txt` analysis

Site Readiness checks your existing `robots.txt` file and provides an AI-optimized version to ensure all major AI crawlers can access and understand your content structure. If a rule is unintentionally blocking a crawler you want to allow, the diagnostic flags it directly.

## `llms.txt` and `llms-full.txt`

* **`llms.txt`** — a lightweight configuration file that helps AI models understand your site's structure and key content. It points AI systems at your best content instead of leaving them to guess.
* **`llms-full.txt`** — a more detailed version of `llms.txt`, providing comprehensive content for AI models to understand your business in depth.

Both are generated automatically based on your current site setup.

### How do I generate or update these files?

Click the **Generate** button for each file. Qwairy auto-generates the content based on your existing site setup. You can then preview and download the files, and re-generate them any time your site structure changes.

## Related pages

* [Crawler Analytics](/documentation/measure/crawler-analytics) — see which crawlers actually visited, and how often.
* [Site Diagnostics](/documentation/optimize/site-diagnostics) — per-page technical, content, and AEO health.
* [Which AI crawlers visit my site?](/use-cases/technical/which-ai-crawlers-visit) and [Is my site optimized for AI crawlers?](/use-cases/technical/site-optimized-for-ai-crawlers) — use-case walkthroughs.