Your Runbook Has No Deadlines: Budgeting Every On-Call Step Before the Page Goes Late

说明与用途

文章说明如何为每个 runbook 步骤添加明确的时间预算，以防止事件超过 SLA 限制。

问题

Runbooks often lack explicit time budgets for each step, causing incidents to overrun their escalation SLAs.

结果

Runbooks with explicit time budgets reduce incident duration and align escalation with SLAs.

工作流

• Define a time budget, stop condition, and timeout action for each runbook step.
• Sum the declared budgets and compare the total plus escalation hold against the SLA.
• If the total exceeds the SLA, adjust budgets or steps before incidents occur.
• Use a validator script to enforce required fields and budget limits.
• Review and argue the budgets in a code‑review style to ensure defensibility.

证据

Most runbooks I read fail for a boring reason: nothing in them has a deadline.

来源

Treat each runbook step as a small contract with three fields: a time budget, a stop condition, and a timeout action.

来源

Worst case here is 120 + 90 + 300 + 240 = 750 seconds, which leaves 150 seconds of slack against a 900-second target.

来源

The script is small on purpose: it validates required fields, sums the declared budgets, and refuses to pass when the total plus the escalation hold exceeds the SLA you pass in.

来源