> ## Documentation Index
> Fetch the complete documentation index at: https://docs.qwairy.co/llms.txt
> Use this file to discover all available pages before exploring further.

# Cloudflare Logpush

> Configure a bounded Cloudflare Logpush HTTP job for Crawler Analytics.

Use Cloudflare Logpush to send a filtered HTTP Requests dataset to Crawler Analytics.

## Before you start

You need:

* a Cloudflare Enterprise zone with Logpush access;
* permission to create the Logpush job;
* a Cloudflare API token with `Logs Write` permission to apply the required upload bounds;
* access to **Measure > Crawler Analytics > Settings**;
* a secure place to retain the one-time connector secret.

## Create and protect the key

1. In Crawler Analytics settings, select **Cloudflare Logpush**.
2. Select **Create Key**.
3. Enter a name and the IANA time zone used for daily snapshots.
4. Copy the secret when it appears. Qwairy shows only its prefix later.
5. Keep it in the Logpush destination header configuration. Do not place it in an ordinary query parameter, source control, or client-side code.

To rotate the key, create a replacement, update the Logpush destination, validate delivery, and then delete the previous key. An integration can have up to five active keys.

## Create the Logpush job

1. In Cloudflare, open **Analytics & Logs > Logs > Add Logpush job** for the target zone.

2. Select the HTTP Requests dataset and an **HTTP destination**.

3. Use this destination configuration, replacing the placeholder with the one-time Qwairy secret:

   ```text theme={null}
   https://www.qwairy.co/api/v1/logs/cloudflare?header_Authorization=Bearer%20YOUR_CRAWLER_KEY
   ```

   Cloudflare converts `header_Authorization` into an HTTP header. The space after `Bearer` must remain encoded as `%20`.

4. Select only these fields:

   ```text theme={null}
   ClientRequestHost
   ClientRequestMethod
   ClientRequestPath
   ClientRequestUserAgent
   EdgeResponseStatus
   EdgeStartTimestamp
   ```

5. Keep the timestamp format as RFC 3339.

6. Before enabling the job, use the Cloudflare Logpush API to set:

   ```json theme={null}
   {
     "max_upload_bytes": 5000000,
     "max_upload_records": 10000
   }
   ```

   The default upload size is outside Qwairy's bounded route contract.

7. Add this job filter. It deliberately over-matches several crawler families; Qwairy's registry performs the final classification.

   ```json theme={null}
   {"where":{"or":[{"key":"ClientRequestUserAgent","operator":"contains","value":"Bot"},{"key":"ClientRequestUserAgent","operator":"contains","value":"bot"},{"key":"ClientRequestUserAgent","operator":"contains","value":"User"},{"key":"ClientRequestUserAgent","operator":"contains","value":"MistralAI"},{"key":"ClientRequestUserAgent","operator":"contains","value":"Notebook"},{"key":"ClientRequestUserAgent","operator":"contains","value":"Google-Agent"},{"key":"ClientRequestUserAgent","operator":"contains","value":"meta-"},{"key":"ClientRequestUserAgent","operator":"contains","value":"Meta-"},{"key":"ClientRequestUserAgent","operator":"contains","value":"spider"},{"key":"ClientRequestUserAgent","operator":"contains","value":"omgili"},{"key":"ClientRequestUserAgent","operator":"contains","value":"Extended"}]}}
   ```

8. Keep sampling at 100% for the filtered dataset, then enable the job.

See [Cloudflare's HTTP destination guide](https://developers.cloudflare.com/logs/logpush/logpush-job/enable-destinations/http/) for the current dashboard and API controls.

## Test delivery

Cloudflare validates the destination with a gzip-compressed `{"content":"tests"}` payload. The Qwairy route accepts that challenge, but it does not create a crawler aggregate.

After validation, request a safe path on the monitored zone with a recognized crawler User-Agent:

```bash theme={null}
curl --head --user-agent "GPTBot/1.0" "https://www.example.com/docs/crawler-check"
```

Check the Logpush job for a successful upload. Qwairy moves the connector from pending after an accepted crawler row reaches the rollup pipeline.

## Limits, sampling, and cost

* Each Logpush upload is limited to 5,000,000 uncompressed bytes and 10,000 records.
* The pre-filter reduces delivery volume but intentionally includes some rows that Qwairy later discards.
* A Cloudflare sampling rate below 100% produces partial Qwairy counts.
* Logpush delivery and retained provider logs may carry Cloudflare charges. Review current pricing for the target zone.
* Shared daily ingestion ceilings also apply.

## Troubleshooting

* **Destination validation fails**: confirm the `%20` encoding, current secret, HTTPS endpoint, and required upload bounds.
* **401**: verify that the destination still sends `Authorization: Bearer YOUR_CRAWLER_KEY` and that the key belongs to Cloudflare Logpush.
* **400 with too many logs**: set `max_upload_records` to `10000` before enabling the job.
* **413**: set `max_upload_bytes` to `5000000` and check compression or decoded payload size.
* **Qwairy stays pending**: verify the six selected fields, the User-Agent filter, 100% sampling, and a current request to the configured hostname.
* **Counts are lower than Cloudflare logs**: compare the same filter and sampling scope, then check Qwairy exclusions and delivery-continuity warnings.

## Related page

* [Crawler Analytics](/documentation/measure/crawler-analytics)