A run history cannot show you a run it never created

说明与用途

运行历史记录无法捕捉未创建的事件，导致可靠性指标失真。通过记录事件、接收确认和发送结果，分类七种结果，并使用 Wilson 区间报告无声失败率，可获得准确的可靠性指标。

问题

Event‑driven systems can report a successful run even when the work never actually happened, because logs only record transport success and cannot capture silent failures or missing runs.

结果

Implementing this reconciliation framework exposes silent failures that run histories hide, enabling accurate reliability metrics and preventing misleading claims of zero failure rates.

工作流

• 1. Emit an event with a unique run_id and metadata (wf, platform, seq, fired_at).
• 2. Each workflow step posts a receipt to a controlled destination, echoing the event and adding step info; the destination stamps received_at.
• 3. The controller records the send outcome (accepted, refused, error type).
• 4. Reconcile by matching receipts to the original event’s run_id and comparing against expected outputs.
• 5. Classify each event into one of seven outcomes based on receipt count and send status.
• 6. Compute silent failure rate as (missed + partial) / (runs accepted and expected to produce output).
• 7. Report the rate with denominator, Wilson interval, as‑of date, workload, and whether the figure is live.
• 8. Publish all seven outcome counts for transparency.
• 9. Use L1, L2, or L3 conformance levels to determine measurement rigor.

证据

A run history cannot show you a run it never created

来源

Events refused at send are excluded from the numerator and from the denominator, and reported separately.

来源

Use the Wilson score interval, not the normal approximation.

来源