My Linter Scored 26% on Its Home Corpus and 0% on Docs — the Calibration Rule I Ship With Now

说明与用途

一个通过自测但在真实代码和文档上误报率高的 linter。引入三步门控：真实代码无 HIGH、MEDIUM 归一化为单类、无输入返回专用退出码。

问题

A linter validator that performed well on its own test fixtures but produced high false positives on real code and documentation, revealing a calibration issue.

结果

The new calibration gate reduced false positives, ensuring validators only publish when they have zero HIGH findings on real code, a single MEDIUM class, and proper handling of no input.

工作流

• Run validator on real, non‑author corpus
• Check that HIGH=0; if not, reject package
• Collapse MEDIUM findings into one rule class; if multiple, reject
• Ensure NO_INPUT returns exit code 3 and NO_INPUT verdict
• Add counters for unknown licenses and other false positives
• Publish only if all gates pass

证据

My cron validator parses schedules out of five different formats: crontab, GitHub Actions schedule, systemd .service/.timer, and a JSON job file.

来源

The rule did not have a dialect problem. It had a lexing problem, and I had been about to fix it in the wrong place.

来源

1. Real corpus, HIGH = 0.

来源

1. Zero input gets its own exit code.

来源