Loading page...Загружаем страницу...正在加载页面...
AtlasRepo

A Testing Strategy for Agent Patches: Property Checks, Fixtures, and a Flaky Freeze

说明与用途

三层门控(属性检查、冻结固定件、消除不稳定测试)可防止 AI 生成的补丁中隐藏的错误。

问题

Agent-generated patches often pass unit tests but break production invariants, causing costly failures that unit tests miss.

结果

Agent patches that pass the gate exhibit robust invariants, consistent data handling, and stable test outcomes, reducing production bugs.

工作流

  • 1. Run property-based tests (e.g., Hypothesis) to verify invariants across generated inputs.
  • 2. Freeze and hash realistic fixtures; compare hashes in CI to detect unintended data changes.
  • 3. Execute flaky freeze script to detect tests that fail inconsistently; quarantine flaky tests and require separate fixes.

证据

Agent patches need a stronger contract. They need properties that hold across the whole input space.

来源

Commit the output. Compare it in CI. If a fixture changes, the diff must be explicit and reviewed.

来源

  • 记录日期: 2026-09-07T17:52:27.615Z