Code review can't keep up with agents. Gates can.

说明与用途

智能变更门通过对受保护文件的更改进行指纹识别并要求评估报告，确保在合并前验证LLM行为，防止回归。

问题

Rapid agent-generated code changes make traditional line-by-line code review impractical, risking unnoticed model behavior changes.

结果

The gate prevented a regression where a context builder change caused the judge to miss all failures, ensuring that any change to model behavior is validated before merging.

工作流

• Declare protected paths and stages in a policy file.
• On each PR, compute a SHA-256 fingerprint of the diff of protected files plus affected stages.
• Require an eval report JSON that lists the fingerprint, dataset, baseline, candidate, and results for each affected stage.
• If the eval passes, the PR is approved; otherwise it is blocked and the coding agent receives a work order to fix the issue.
• Re-run the gate after any new commit to the protected files.

证据

The model had not changed, and neither had the prompt template. Our evals ran when one of those two changed, so nobody had run the golden dataset, and when the failures went missing we did not know which change was responsible.

来源

The eval gate is the most important idea in this post: a CI check that knows which files can change model behavior.

来源

Step 3: require an eval report for that fingerprint.

来源