Properties, Fixtures, and a Flaky Freeze: A Three-Layer Gate for Agent Patches

说明与用途

三层门控（属性检查、固定测试和不稳定冻结）通过防止过拟合、边界幻觉和非确定性，提高代理编写补丁的可靠性。

问题

Agent patches often pass tests that only cover the prompt examples, leading to overfitting, edge hallucination, and nondeterminism.

结果

The gate catches edge‑case failures and ensures patches generalize beyond the prompt examples, improving merge confidence.

工作流

• 1. Define property checks that generate random inputs and assert invariants.
• 2. Implement a minimal property harness with a fixed‑seed RNG for reproducibility.
• 3. Write property functions (e.g., round‑trip and additive for parse_duration).
• 4. Run the harness with a fixed number of trials (e.g., 2000).
• 5. If any trial fails, report the failing input and exit with non‑zero status.
• 6. Use the flaky freeze to ensure the gate signal is clean before merging.

证据

Property checks, fixtures, and a flaky freeze. Each layer answers a different question.

来源

The fixed seed is deliberate. A gate that fails on trial 731 today and trial 402 tomorrow is useless for merge decisions.

来源