Anchored Review Findings: Reject Model Comments That Cite No Diff Line

Description and use

Validating model findings against real diff lines reduces reviewer effort.

Problem

Model reviews often produce findings that reference lines or code not present in the actual diff, wasting maintainers’ time.

Outcome

The gate removes all findings that do not anchor to a real added line, reducing reviewer effort and improving review quality.

Workflow

• Export a zero‑context diff with git diff -U0 limited to source files.
• Send the diff and a review prompt to an OpenAI‑compatible chat endpoint, receiving JSON output.
• Strip any code fences from the model’s response.
• Run the anchor_gate.py script to verify each finding’s file, line number, and quote against the diff; drop findings that fail the checks.
• Proceed with the remaining findings for human review.

Evidence

A review with ten findings and four invented anchors is worse than no review. Maintainers spend the first ten minutes disproving fiction.

Source

That check is a script. It verifies file path, line number, and quoted text against the diff. Findings that fail are dropped before any human reads them.

Source

The workflow below keeps the normal open‑source loop: reproduce the bug, patch it, test it. It adds one deterministic check to the review step.

Source