Skip to main content
GET
Get crawler analytics
Retrieve AI crawler activity observed through the brand’s configured log integration. The endpoint returns daily aggregates and page-level observations. It never returns raw log events or human traffic. All occurrence totals, crawler breakdowns, and page rankings use the public set of supported observable AI-specific User-Agent identities. The ingestion registry can recognize additional identities for classification and compatibility, but the endpoint excludes them from its public analytics output. Classification matches the received User-Agent token; it does not verify source IP or reverse DNS.
This endpoint requires an active Business, Enterprise, or Agency Business plan, or an explicit Crawler Analytics feature override for the team. Results are observational: check coverage before interpreting counts or comparing periods.
string
required
Bearer token. Example: Bearer qw-api-xxx

Path parameters

string
required
Unique identifier of the brand.

Query parameters

string
default:"30d"
Inclusive analytic-day preset in the integration’s IANA time zone: 24h is the current analytic day, 7d is that day plus the preceding six, and 30d is that day plus the preceding 29. 24h is not a rolling 24-hour window.
string
default:"all"
Page-state filter: all, new, hot, or neglected.
number
default:"1"
One-based page number. Maximum: 10000.
number
default:"50"
Page size. Maximum: 100.

Page-state semantics

  • new: the first retained page observation falls within the last 7 days, including the boundary (firstSeenAt >= now - 7 days).
  • hot: the page is in the top decile by supported-crawler occurrences over the current 7 analytic days, has activity on at least 2 of those days, and is not neglected.
  • neglected: the most recent retained page observation is strictly older than the 30-day boundary.
new and hot can coexist. neglected is exclusive. The hot window is independent of the requested range.

Response

boolean
Indicates whether the request succeeded.
object
Brand id, name, and domain.
boolean
Whether the integration is active, not disabled, and has at least one active non-revoked key.
string | null
Current integration lifecycle, or null when no integration exists.
string | null
Current integration health, or null when no integration exists.
object
Requested and effective dates, observed and verified bucket counts, truncation state, and the asOf timestamp.
object
number | null
Observed occurrences from the supported public crawler set in the effective range. A number is returned only when at least one supported public rollup has a positive request count; otherwise the value is null, not a measured zero. With incomplete ingestion, a numeric value can still be partial.
number
Occurrences associated with a retained page identity.
number
Requests that returned a redirect.
number
Requests that returned 404.
number
Other observed error responses.
array
Daily totals, crawler breakdowns, error totals, and coverage metadata.
array
Totals by crawler in the supported public set, including its display name, bot type, and error breakdown.
array
Page-by-crawler daily rollups retain 30 analytic days. Durable page identities preserve first seen, last seen, and lifetime count beyond that window. Query strings and fragments are removed; sensitive-looking or overlong paths can be replaced with an integration-scoped one-way pseudonym.
object
One-based page, limit, totalItems, totalPages, and hasMore.
boolean
Always false. Raw log events are not exposed by this endpoint.
boolean
Always false.

Synthetic request

Interpret missing or incomplete data

  • Treat totalOccurrences: null as not measured. Do not convert it to zero.
  • Check coverage.dataStatus, countPrecision, and deliveryContinuity before comparing periods.
  • Use range.startDate and range.endDate for the effective comparison window.
  • Do not build a public URL from a page where isPseudonymized is true.

Errors

See Crawler Analytics for setup and the product interface, and Rate limits for retry behavior.