OpenAPI linters: which one is the best in a real test?

Description and use

Benchmarking five OpenAPI linters revealed Redocly CLI as the fastest, most stable, and most consistent in error reporting across bundled and split specifications.

Problem

Choosing a reliable OpenAPI linter that performs consistently across different API description layouts and sizes.

Outcome

Redocly CLI was the fastest and most stable linter, handling both bundled and split specs without crashes, and providing consistent error reporting. Other linters either timed out, crashed, or produced inconsistent results due to reference handling or rule set differences.

Workflow

• 1. Clone the public benchmark repo
• 2. Install latest releases of each linter
• 3. Run each linter on each spec five times on a GitHub-hosted runner
• 4. Record median lint times and raw outputs
• 5. Split bundled specs with redocly split and bundle split specs with redocly bundle
• 6. Compare error, warning, and info counts across layouts
• 7. Analyze rule sets and error meanings
• 8. Publish results and raw data in the repo

Evidence

The setup is simple: latest release of each tool, default rules, one GitHub-hosted runner, one tool at a time, five runs each, median reported.

Source

Redocly CLI finished first at 1.26 s, Vacuum second at 2 s and Speakeasy third at 3.3 s. Spectral came last, far behind at 12 s. Scalar CLI crashed.

Source

Redocly CLI draws the flattest line and the spec size affects it the least.

Source

Redocly CLI was the only tool that reports a problem written into the OpenAPI specification: almost all of its errors are schemas where nullable: true has no effect.

Source