> ## Documentation Index > Fetch the complete documentation index at: https://docs.qwairy.co/llms.txt > Use this file to discover all available pages before exploring further. # Why does Google rarely appear in Crawler Analytics? > Understand which Google crawlers can appear in Crawler Analytics, why Googlebot is excluded, and which data shows your visibility in Google's AI features. ## Direct answer Few Google requests in **Measure > Crawler Analytics** usually reflect how Google crawls, not low visibility in Google. Google Search, including AI Overviews and AI Mode, relies on Googlebot, the same crawler as traditional search. Crawler Analytics counts AI-specific crawlers only, so it excludes Googlebot: a Googlebot request does not show whether the page will serve a traditional result or an AI feature. The Google identities that Crawler Analytics can count fetch pages for narrower uses, often on a user's request, so low volumes are expected. To measure whether your brand appears in Google's AI answers, use answer monitoring rather than crawler counts. ## Data required Use Crawler Analytics occurrences by User-Agent identity for the selected period, with the connector's coverage state. For Google's AI features, use your monitored prompts and the answers recorded for the Google providers selected in your workspace. For Googlebot activity, use your server or CDN logs and the crawl reports in Google Search Console. ## Workflow **1. Confirm collection** Check that the connector is connected, that the coverage state is complete for the period, and that no upstream filter removes Google User-Agents you expect to see. **2. Identify the Google rows** Compare the observed identities with the table in [How Google crawlers map to Crawler Analytics](#how-google-crawlers-map-to-crawler-analytics). Only identities from the public set appear in totals and tables. **3. Measure visibility in Google's AI features** In **Monitor > Prompt Tracking**, review the answers recorded for Google providers. Mentions and citations show whether your brand appears in those answers. **4. Review Googlebot crawling separately** Use Google Search Console crawl statistics or your raw server logs to review how Googlebot crawls the site. Treat this activity as search crawling, not as an AI-specific signal. ## How Google crawlers map to Crawler Analytics Google documents these identities. The column on the right describes what Crawler Analytics does with each one. | Identity | What Google documents | In Crawler Analytics | | - | - | - | | `Googlebot` | Crawls for Google Search. Google states that AI is built into Search, and that `robots.txt` rules for Googlebot control how a site is crawled for Search, including its AI features. | Excluded. Its requests cannot be attributed to traditional results or to AI features. | | `Google-Extended` | A `robots.txt` control token for Gemini training and grounding. It has no separate HTTP User-Agent: Google crawls with its existing User-Agents. | Never observed in logs. Excluded as a robots policy token. | | `Google-GeminiNotebook` | Fetches individual URLs that Gemini Notebook users add as sources to their projects. `Google-NotebookLM` is its former name. | Counted. `Google-NotebookLM` is accepted as an alias. | | `Google-CloudVertexBot` | Crawls requested by site owners who build Vertex AI agents. It has no effect on Google Search. | Counted. | | `Google-Agent` | Used by agents hosted on Google infrastructure to navigate the web and act on a user's request. | Recognized on receipt, but outside the public identity set, so it does not appear in totals or tables. | | `GoogleOther` | A generic crawler used by various Google product teams. Its requests do not affect a specific product. | Excluded. Its purpose cannot be attributed to an AI feature. | Sources: [AI features and your website](https://developers.google.com/search/docs/appearance/ai-features) (updated 10 December 2025), [Google's common crawlers](https://developers.google.com/crawling/docs/crawlers-fetchers/google-common-crawlers) (updated 15 October 2026), and [Google's user-triggered fetchers](https://developers.google.com/crawling/docs/crawlers-fetchers/google-user-triggered-fetchers) (updated 19 August 2026). Verified on 30 October 2026. ## Interpretation A low Google count in Crawler Analytics does not measure your visibility in AI Overviews, AI Mode, or Gemini. Those features can use content that Googlebot already crawled for Search, so an AI answer does not require a new, identifiable request in your logs. Google rows that do appear describe specific uses. `Google-GeminiNotebook` shows that users added your pages as sources in Gemini Notebook. `Google-CloudVertexBot` shows crawls requested for Vertex AI agents. Neither represents Google Search as a whole. ## Possible next actions * To track whether your brand appears in Google's AI answers, monitor prompts with the Google providers available in your workspace and compare mention and citation rates over time. * To check that Google can crawl important pages, review Search Console crawl statistics and the status codes Googlebot receives in your server logs. * To change how Google may use your content for Gemini training and grounding, review the `Google-Extended` rules in your `robots.txt` with **Optimize > Site Readiness**. ## Limitations Crawler Analytics classifies the User-Agent string it receives and does not verify the requester's IP address or reverse DNS. A User-Agent can be spoofed. Google can rename or add identities. This page describes Google documentation on the verification date above; recheck the linked sources before acting on a specific identity. ## Related pages > **[Crawler Analytics](https://docs.qwairy.co/documentation/measure/crawler-analytics.md)** > > Read crawler data and the public identity set. > **[Which AI crawlers visit my site?](https://docs.qwairy.co/use-cases/technical/which-ai-crawlers-visit.md)** > > Review crawler occurrences by identity and page. > **[robots.txt and llms.txt](https://docs.qwairy.co/use-cases/technical/robots-txt-llms-txt.md)** > > Review crawler directives. > **[Crawled but not cited](https://docs.qwairy.co/use-cases/technical/pages-crawled-but-never-cited.md)** > > Compare crawler and citation observations. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.