Runbook Rot: Three Fingerprints of Stale CI Docs and the Check That Catches Each

说明与用途

CI 自动化检查 runbook 的新鲜度，检测时间戳差距、孤立引用和程序动词，防止夜间错误。

问题

Stale CI runbooks cause costly errors when the documented pipeline no longer matches the live environment.

结果

Runbooks are now automatically validated for freshness, reducing 3 a.m. manual interventions and preventing costly errors.

工作流

• Extract last content edit of the doc and compare it to the last edit of its subject to detect timestamp gaps.
• Extract backticked identifiers from the runbook and verify each against the repository to find orphaned referents.
• Search for procedural verbs (e.g., "ask", "ping", "check") and flag them as work tickets.
• Add a front‑matter header with as_of, scope, cadence, and owner to each runbook.
• Run a CI job that calculates lag = now() - as_of and fails the build if lag exceeds cadence.
• When a failure occurs, bump as_of only after a human verifies the steps against the live system.
• Repeat the checks on a scheduled cadence to maintain freshness.

证据

The document's last meaningful edit predates the last change to the thing it describes.

来源

The check: extract identifiers from the doc and verify each against ground truth

来源

The check is a grep, and every hit is a work ticket

来源

Then a one-page CI job computes lag = now() - as_of against cadence and fails the build at 2× expected interval

来源