Your scraper says SUCCEEDED but the data is empty: three ways to catch it

Description and use

Scrapers that finish with SUCCEEDED may output empty or incomplete data. The article outlines three ways to detect such failures: enable Apify’s free monitoring, compare the latest run to historical medians, and use ready‑made Actors for data completeness checks.

Problem

Scraper runs report success while output data is empty or incomplete, causing undetected failures.

Outcome

By enabling free monitoring, comparing runs to historical medians, and using ready‑made Actors, silent scraper failures are detected early, reducing data quality issues.

Workflow

• 1. Enable Apify’s free Monitoring tab for each Actor or task, setting alerts on result-count thresholds and run status.
• 2. If fixed thresholds are insufficient, run a Node.js script that compares the latest run to the median of recent successful runs, sampling the first 200 items and checking field completeness.
• 3. Optionally use third‑party Actors such as Output Completeness Monitor, Dataset Drift Monitor, or the author’s Scraper Output Guard for ready‑made checks.

Evidence

Each Actor or task has a Monitoring tab. You can set alerts on result-count thresholds and run status.

Source

The guide on GitHub has a small Node.js script that does this with three API calls (list runs, get dataset metadata, fetch the first 200 items).

Source

There are third-party Actors with similar checks, such as Output Completeness Monitor and Dataset Drift Monitor.

Source