Is that visitor a person or an agent? CrawlProof now lets them say so

Description and use

CrawlProof lets visitors declare themselves as human or agent, improving visitor count accuracy and eliminating inflated bot traffic.

Problem

Site owners cannot reliably distinguish human visitors from headless scraper bots because bots mimic real browsers, leading to inflated visitor counts and inaccurate analytics.

Outcome

By allowing visitors to declare themselves, CrawlProof accurately separates human traffic from bots, preventing inflated visitor counts and improving analytics reliability.

Workflow

• 1. Register actors in CrawlProof with email and kind (human or agent).
• 2. For agents, optionally specify an operator email.
• 3. Generate one or more cpa_ tokens for each actor.
• 4. Agents set a header (Crawlproof-Actor: cpa_…) on every request using Playwright or Puppeteer.
• 5. Humans add the token to the URL once per site (https://example.com/?crp_actor=cpa_…).
• 6. CrawlProof records the declaration and updates visitor counts accordingly.
• 7. Use crawlproof stats or API to view declared actors and reconcile with measured data.

Evidence

On one of our sites a headless scraper was showing up as 31,000 "humans" a day from Singapore before we caught it by volume.

Source

CrawlProof's tracker now accepts a declaration: this visit is a person, or this visit is an agent, and here is who.

Source

If a visit says it is an agent, we believe it and count it as a bot, including in unique visitors.

Source