Your Scraper Didn’t Fail. It Just Started Lying. So I Built Molt

Описание и применение

Molt обнаруживает и исправляет скрытые ошибки в веб‑скраперах, сравнивая результаты с базой, диагностируя изменённые поля и генерируя точные инструкции для исправления, при этом требуя ручного одобрения.

Задача

Scrapers can succeed technically while producing incorrect data, leading to silent failures that go unnoticed by standard monitoring checks.

Результат

Molt enabled automated detection of data drift, diagnosis of affected fields, generation of precise heal prompts, and a human‑in‑the‑loop approval gate, closing the loop from anomaly detection to verified data recovery.

Рабочий процесс

• Run scraper and snapshot output
• Compare snapshot to baseline to detect drift
• Diagnose affected fields and quantify drift
• Generate a targeted heal prompt describing changes
• Human reviews the heal prompt and approves or rejects
• Apply heal via Bright Data CLI (bdata scraper heal)
• Re-run scraper to verify data recovery

Доказательства

The job was still successful. HTTP was still 200. The row count was still 60. And download_count = 0 is a perfectly plausible number.

Источник

Molt treats scraper breakage more like an SRE incident than a failed request.

Источник