Skip to main content
Open Optimize > Site Readiness to review access signals for the monitored domain.

AI Readiness Score

The AI Readiness Score summarizes the checks below, from Critical to Excellent, with the number of issues to fix and optimizations available. Select Re-check to fetch robots.txt, llms.txt, and sitemap.xml again. A re-check is available once per day. When at least one file cannot be fully verified, the page shows Analysis incomplete instead of a score. This does not mean that the file is missing or invalid. When a file is not found at the checked URL but exists on the other host, with or without www, its card shows that URL and Use this URL. robots.txt rules found there apply only to that host. Only Owners and Managers can change a file URL, including with Use this URL.

Crawler and robots-policy access

The access table checks configured crawler and robots-policy identities against the site’s current robots.txt rules. Each identity is Allowed, Blocked, has Mixed access when some paths are blocked or allowed again, or is Unknown when access was not analyzed or robots.txt could not be read. Some entries are observable crawler User-Agents, such as GPTBot. Others are policy tokens, such as Google-Extended, that control permitted uses of content fetched by another crawler. A policy-token result cannot produce a Crawler Analytics occurrence or last-seen timestamp. The result is a robots-policy observation. It does not show that a provider has crawled, indexed, cited, or will use the site. Only the supported observable User-Agent subset can appear in Crawler Analytics.

robots.txt

The robots.txt check reads the public file and reports whether it is present and valid. Review the raw file before changing it, especially when rules differ by user agent or path. If Qwairy cannot retrieve the file, check the URL, status code, redirects, firewall, and authentication requirements.

llms.txt

The llms.txt check reports whether the file is present and valid. When it is missing, copy the generated template, customize it, and upload it to the root of your site, for example example.com/llms.txt. Support and interpretation vary across AI systems. Publishing the file does not guarantee discovery or citation. Review every URL and description in the template before placing it on the public site.

sitemap.xml

The sitemap check reports whether sitemap.xml is present and valid. A sitemap helps discovery; it does not guarantee crawling.
  1. Confirm the monitored domain and protocol.
  2. Inspect the current robots.txt response.
  3. Review user-agent-specific rules.
  4. Check the proposed llms.txt content, if used.
  5. Coordinate deployment with the site owner.
  6. Re-run the check after the public files change.