A PR Review Gate That Fails Closed: Free Models in CI

Description and use

A PR review gate that fails closed uses free AI models to classify findings after running tests, blocking merges on BLOCK findings and allowing merges otherwise, providing automated verification for small open‑source projects.

Problem

Reviewers are rarely tested, leading to bugs slipping through; free AI models can hallucinate and block good code, so a deterministic gate that fails closed is needed.

Outcome

The gate blocks merges when the model flags BLOCK findings, allows merges when only ASK or NIT findings are present, and logs the review for human inspection.

Workflow

• Extract the diff with gh pr diff and git diff --stat
• Run the test suite and capture output
• Generate a structured review prompt from diff, stats, and test log
• Send prompt to a free model (e.g., MonkeyCode) and parse output
• Fail the gate if any BLOCK findings are present

Evidence

Free models hallucinate. They praise broken code. They block good code.

Source

The gate runs four steps. Each step writes a file. Each file feeds the next step.

Source