My coding agent rewrote the test to agree with its bug. So I built a gate that says INCONCLUSIVE.

Description and use

AdversaryGate blocks false merges by running baseline tests against patched code, returning BLOCK on regression and INCONCLUSIVE when measurements are missing.

Problem

Coding agents can rewrite tests to match buggy code, causing CI to green while the bug persists.

Outcome

AdversaryGate prevents false merges by ensuring baseline tests pass on patched code, returning BLOCK when a regression is detected and INCONCLUSIVE when measurements are missing.

Workflow

• 1. Agent submits patch that changes implementation and test.
• 2. AdversaryGate copies the patch tree and restores baseline tests.
• 3. AdversaryGate runs baseline tests against patched code.
• 4. If any baseline test fails, AdversaryGate returns BLOCK; if all pass, returns MERGE; if tests are unmeasured, returns INCONCLUSIVE.

Evidence

AdversaryGate notices the patch changed a test the baseline already had. It copies the patch's tree, puts back the baseline's copy of the tests (helpers included), and runs them against the new code:

Source

The test that runs is the one that existed before the agent touched anything. Bending it doesn't help.

Source