Split the Flake Before You Freeze It

说明与用途

将 flaky 测试拆分为属性核心、fixture 带和时间尾部，只冻结时间尾部，保持核心属性为必需测试。

问题

Agent patches often make tests intermittent, but freezing the entire test hides the underlying business rule and envelope, leading to fragile CI.

结果

Flaky tests are now split into deterministic core and freezeable timing tail, improving CI stability and test intent preservation.

工作流

• Reproduce the flake once with a tight command to confirm intermittency.
• Classify the failing layer by reading the assertion: core, fixture, or timing tail.
• Extract the core into a separate property test to ensure determinism.
• Lock fixture inputs (hash or copy) and fail CI if agent commits change them without a ledger note.
• Write a freeze ledger entry only for the timing tail, including witness command, remaining property, and expiry.
• Fail the gate if a freeze marker lacks a live property test.
• Use a pytest hook to enforce that only timing tail freezes are allowed and that remaining_property is present.

证据

A freeze on a whole test is a deleted oracle.

来源

Freeze records how the flake was produced, under which patch, and which property remains in the gate.

来源