Code review can't keep up with agents. Gates can.

Description and use

The Intelligence Change Gate enforces evidence-based evaluation for protected file changes, blocking merges until a passing eval report is provided, thus safeguarding LLM behavior.

Problem

Rapid agent-generated code changes make traditional line-by-line code review impractical, risking unnoticed model behavior changes.

Outcome

The gate prevented a regression where a context builder change caused the judge to miss all failures, ensuring that any change to model behavior is validated before merging.

Workflow

• Declare protected paths and stages in a policy file.
• On each PR, compute a SHA-256 fingerprint of the diff of protected files plus affected stages.
• Require an eval report JSON that lists the fingerprint, dataset, baseline, candidate, and results for each affected stage.
• If the eval passes, the PR is approved; otherwise it is blocked and the coding agent receives a work order to fix the issue.
• Re-run the gate after any new commit to the protected files.

Evidence

The model had not changed, and neither had the prompt template. Our evals ran when one of those two changed, so nobody had run the golden dataset, and when the failures went missing we did not know which change was responsible.

Source

The eval gate is the most important idea in this post: a CI check that knows which files can change model behavior.

Source

Step 3: require an eval report for that fingerprint.

Source