Don't trust "Done": a two-model review gate for AI coding agents

说明与用途

使用哈希锁定合同的双模型完成门检测到所有真实缺陷，并防止接受不完整修复，仅在四轮中耗时五分钟审阅者时间。

问题

AI coding agents claim completion without reliable verification, leading to undetected bugs and false confidence.

结果

The two-model gate successfully identified all real defects, prevented acceptance of incomplete fixes, and required only about five minutes of reviewer time across four rounds, with the author’s repair effort exceeding reviewer effort.

工作流

• 1. Define binary acceptance criteria and not-done conditions in CONTRACT.md and lock with SHA-256 hash.
• 2. Author generates code and writes a completion report claiming DONE.
• 3. Reviewer model runs in read-only sandbox, computes contract hash, evaluates each criterion, cites file:line evidence, and lists findings.
• 4. Reviewer reports findings and overall verdict (DONE/NOT DONE).
• 5. Author addresses findings, updates code and completion report.
• 6. Reviewer re-evaluates in next round, focusing only on previously raised findings.
• 7. Repeat until reviewer returns DONE or escalation threshold is reached.

证据

The reviewer runs with a read-only sandbox and no network.

来源

The reviewer returned DONE. It did not take my word for the fix: it reported building in-memory tab lists for three cases...

来源

Four review rounds plus the control came to roughly 790k tokens, about 77% of them cache hits...

来源