Is that visitor a person or an agent? CrawlProof now lets them say so

Описание и применение

CrawlProof позволяет посетителям объявлять себя как человек или агент, что повышает точность подсчёта посетителей и устраняет завышенные показатели из-за ботов.

Задача

Site owners cannot reliably distinguish human visitors from headless scraper bots because bots mimic real browsers, leading to inflated visitor counts and inaccurate analytics.

Результат

By allowing visitors to declare themselves, CrawlProof accurately separates human traffic from bots, preventing inflated visitor counts and improving analytics reliability.

Рабочий процесс

• 1. Register actors in CrawlProof with email and kind (human or agent).
• 2. For agents, optionally specify an operator email.
• 3. Generate one or more cpa_ tokens for each actor.
• 4. Agents set a header (Crawlproof-Actor: cpa_…) on every request using Playwright or Puppeteer.
• 5. Humans add the token to the URL once per site (https://example.com/?crp_actor=cpa_…).
• 6. CrawlProof records the declaration and updates visitor counts accordingly.
• 7. Use crawlproof stats or API to view declared actors and reconcile with measured data.

Доказательства

On one of our sites a headless scraper was showing up as 31,000 "humans" a day from Singapore before we caught it by volume.

Источник

CrawlProof's tracker now accepts a declaration: this visit is a person, or this visit is an agent, and here is who.

Источник

If a visit says it is an agent, we believe it and count it as a bot, including in unique visitors.

Источник