Your API returns 200. Your agent still gets garbage

说明与用途

API 可能返回 200 OK 并具有有效的 JSON 模式，但数据可能无效。driftprobe 工具通过检查覆盖率、新鲜度、体积和多样性来检测此类问题，并通过 CI 通知团队。

问题

An API that returns 200 OK and a valid JSON schema can still deliver unusable data, causing an agent to produce incorrect reports without detection.

结果

The driftprobe tool successfully identified a data quality failure that would otherwise go unnoticed by traditional status‑code and schema checks, enabling timely remediation.

工作流

• 1. Record real API calls with a local proxy to learn a contract that captures field presence, freshness, coverage, volume, and distinctness.
• 2. Simulate a source degradation by sending a POST to the proxy’s break endpoint.
• 3. Run the driftprobe check against the learned contract to evaluate the current data against the defined rules.
• 4. Review the check output; a non‑zero exit code triggers CI failure and alerts the team.

证据

An API your agent calls every morning answers 200 OK. The JSON schema has not changed. Every field is still there. But 42 of the 50 products now have "price": null, and updated_at has not moved in months.

来源

Coverage: at least 90% of items carry a usable price. null, "", [], and {} do not count.

来源

node bin/driftprobe.mjs check contracts/*.yaml GET /products x price: only 16% of 50 items carry a value (contract: at least 90%) x updated_at: newest value is 289 days old (contract: 3d) 1 contract(s): 0 ok, 1 broken, 0 could not be checked

来源