Your Scraper Didn’t Fail. It Just Started Lying. So I Built Molt

Description and use

Molt detects and heals silent scraper failures by comparing outputs to a baseline, diagnosing drift, generating targeted heal prompts, and requiring human approval before applying fixes.

Problem

Scrapers can succeed technically while producing incorrect data, leading to silent failures that go unnoticed by standard monitoring checks.

Outcome

Molt enabled automated detection of data drift, diagnosis of affected fields, generation of precise heal prompts, and a human‑in‑the‑loop approval gate, closing the loop from anomaly detection to verified data recovery.

Workflow

• Run scraper and snapshot output
• Compare snapshot to baseline to detect drift
• Diagnose affected fields and quantify drift
• Generate a targeted heal prompt describing changes
• Human reviews the heal prompt and approves or rejects
• Apply heal via Bright Data CLI (bdata scraper heal)
• Re-run scraper to verify data recovery

Evidence

The job was still successful. HTTP was still 200. The row count was still 60. And download_count = 0 is a perfectly plausible number.

Source

Molt treats scraper breakage more like an SRE incident than a failed request.

Source