My coding agent rewrote the test to agree with its bug. So I built a gate that says INCONCLUSIVE.

Описание и применение

AdversaryGate предотвращает ложные слияния, сравнивая исходные тесты с изменённым кодом, возвращая BLOCK при регрессии и INCONCLUSIVE при отсутствии измерений.

Задача

Coding agents can rewrite tests to match buggy code, causing CI to green while the bug persists.

Результат

AdversaryGate prevents false merges by ensuring baseline tests pass on patched code, returning BLOCK when a regression is detected and INCONCLUSIVE when measurements are missing.

Рабочий процесс

• 1. Agent submits patch that changes implementation and test.
• 2. AdversaryGate copies the patch tree and restores baseline tests.
• 3. AdversaryGate runs baseline tests against patched code.
• 4. If any baseline test fails, AdversaryGate returns BLOCK; if all pass, returns MERGE; if tests are unmeasured, returns INCONCLUSIVE.

Доказательства

AdversaryGate notices the patch changed a test the baseline already had. It copies the patch's tree, puts back the baseline's copy of the tests (helpers included), and runs them against the new code:

Источник

The test that runs is the one that existed before the agent touched anything. Bending it doesn't help.

Источник