What to Trust, Revert, and Test in an Agent-Generated Pull Request

Description and use

The workflow identifies and removes out-of-scope changes, adds contract and failure tests, reducing hidden dependency risk.

Problem

Agent-generated PRs often modify unrelated files such as config, auth, or CI workflows, leading to scope creep and hidden dependencies.

Outcome

The workflow isolates scope creep, ensures only relevant changes are reviewed, and enforces contract and failure tests, reducing risk of hidden dependencies.

Workflow

• Sort diff into Trust, Revert, Test buckets
• Run diff_scope.py to flag sensitive files
• Grep for exception handling and default changes
• Revert out-of-scope files and re-run CI
• Add contract test for happy path
• Add failure-injection test for error handling

Evidence

CI is green. The diff also edits the shared config loader and one auth middleware file.

Source

Green CI proves compilation, not intent

Source

Step 1 — Sort the diff into three buckets

Source

Run it against the PR branch: python diff_scope.py origin/main

Source