Your scraper says SUCCEEDED but the data is empty: three ways to catch it

Описание и применение

Скрипты, которые завершаются статусом SUCCEEDED, могут выдавать пустые или неполные данные. В статье описаны три способа обнаружения таких ошибок: включение бесплатного мониторинга Apify, сравнение текущего запуска с историей и использование готовых Actors для проверки целостности данных.

Задача

Scraper runs report success while output data is empty or incomplete, causing undetected failures.

Результат

By enabling free monitoring, comparing runs to historical medians, and using ready‑made Actors, silent scraper failures are detected early, reducing data quality issues.

Рабочий процесс

• 1. Enable Apify’s free Monitoring tab for each Actor or task, setting alerts on result-count thresholds and run status.
• 2. If fixed thresholds are insufficient, run a Node.js script that compares the latest run to the median of recent successful runs, sampling the first 200 items and checking field completeness.
• 3. Optionally use third‑party Actors such as Output Completeness Monitor, Dataset Drift Monitor, or the author’s Scraper Output Guard for ready‑made checks.

Доказательства

Each Actor or task has a Monitoring tab. You can set alerts on result-count thresholds and run status.

Источник

The guide on GitHub has a small Node.js script that does this with three API calls (list runs, get dataset metadata, fetch the first 200 items).

Источник

There are third-party Actors with similar checks, such as Output Completeness Monitor and Dataset Drift Monitor.

Источник