We scanned 1,939 repositories. 95% of projects publishing an OpenAPI spec have no compatibility gate

说明与用途

大多数发布 OpenAPI 规范的项目缺乏自动兼容性检查，但许多项目会主动披露破坏性变更。研究将最初的 95% 误报修正为 67% 的披露率，强调需要可靠的门控和更好的工具集成。

问题

Many OpenAPI‑publishing projects lack automated compatibility checks, risking silent breaking changes for consumers.

结果

The study revealed that while most OpenAPI projects lack automated compatibility gates, many teams proactively disclose breaking changes. The initial claim of 95% of projects flying blind was inflated due to ordering, parallel line, and vocabulary errors; the corrected figure shows 67% of breaking changes are disclosed. The data underscores the need for reliable gates and better tooling integration.

工作流

• Harvest candidate repos via GitHub issue‑search queries for breaking‑change complaints (~2,800 hits).
• De‑noise by requiring ≥2 comments, yielding 1,939 repos.
• Filter for first‑party OpenAPI specs, excluding vendor, node_modules, testdata, etc., leaving 85 repos.
• Check workflow files for known OpenAPI diff tools; only 4 repos used oasdiff.
• For each repo, fetch up to ten consecutive releases, diff adjacent specs, classify changes as consumer‑visible or not.
• Correct for version ordering, parallel maintenance lines, and direction of change to avoid over‑counting.
• Identify and count disclosed breaking changes via release notes, adjusting for vocabulary and length limits.
• Compile results and publish datasets for transparency.
• Share findings and lessons learned with the community.

证据

Of 85 repositories with a genuine first‑party OpenAPI spec, exactly four run any breaking‑change detection at all. That is 95.3% with no compatibility gate

来源

Of the 18 repositories that shipped a consumer‑visible breaking change, 12 disclosed every one of them — 67%.

来源

The largest single case was a project that renamed its entire core resource — every /v1/items* route replaced by /v1/entities* — in one release.

来源