Your API returns 200. Your agent still gets garbage

Описание и применение

API может возвращать 200 OK и валидный JSON, но данные становятся непригодными. Инструмент driftprobe выявляет такие проблемы, проверяя покрытие, свежесть и разнообразие данных, и сообщает об ошибках в CI.

Задача

An API that returns 200 OK and a valid JSON schema can still deliver unusable data, causing an agent to produce incorrect reports without detection.

Результат

The driftprobe tool successfully identified a data quality failure that would otherwise go unnoticed by traditional status‑code and schema checks, enabling timely remediation.

Рабочий процесс

• 1. Record real API calls with a local proxy to learn a contract that captures field presence, freshness, coverage, volume, and distinctness.
• 2. Simulate a source degradation by sending a POST to the proxy’s break endpoint.
• 3. Run the driftprobe check against the learned contract to evaluate the current data against the defined rules.
• 4. Review the check output; a non‑zero exit code triggers CI failure and alerts the team.

Доказательства

An API your agent calls every morning answers 200 OK. The JSON schema has not changed. Every field is still there. But 42 of the 50 products now have "price": null, and updated_at has not moved in months.

Источник

Coverage: at least 90% of items carry a usable price. null, "", [], and {} do not count.

Источник

node bin/driftprobe.mjs check contracts/*.yaml GET /products x price: only 16% of 50 items carry a value (contract: at least 90%) x updated_at: newest value is 289 days old (contract: 3d) 1 contract(s): 0 ok, 1 broken, 0 could not be checked

Источник