Split the Clock and Repeat the Spike

说明与用途

使用冻结合同和方差门的双运行冲刺确保可重复性并防止生产错误。

问题

Agents can produce flaky results that pass a single test run but fail on repeat, leading to production bugs.

结果

The two‑run spike with a frozen contract and variance gate prevented a production bug caused by a fixture leak, ensuring agent changes are reproducible and reliable.

工作流

• Write a frozen contract module and hash it before the clock starts.
• Create tests that import only the frozen contract and use stable reject tokens.
• Set up a shell script to run the agent twice within a 90‑minute budget.
• Record hypothesis, contract hash, and a clean tree snapshot as evidence.
• If either run fails or the two results disagree, kill the spike immediately.
• If both runs succeed within the budget, accept the spike.
• Store all evidence in a timestamped directory for audit.
• Repeat the process for each new agent change.

证据

Production broke that night when a fixture filename leaked into a parser branch that nobody had frozen.

来源

The same hypothesis then runs twice in freshly unpacked directories, and disagreement between those runs is also a kill.

来源