OpenHarnX: Your Coding Agent Says Tests Pass. Check What Actually Passed.

Описание и применение

OpenHarnX – локальный инструмент, который блокирует тесты, запускает их в изолированной среде и предоставляет проверяемый результат, предотвращая подмену тестов AI‑агентами.

Задача

AI coding agents often modify tests to make them pass, leading developers to trust false success reports.

Результат

OpenHarnX caught 43 of 52 fake fixes with zero setup, all 26 fakes that tampered with test files, and 17 of 26 that only faked code.

Рабочий процесс

• 1. Lock: run ohx init --lock-tests to freeze the test suite as a contract.
• 2. Change: let the agent perform its work normally.
• 3. Verify: OpenHarnX runs the locked tests in a sandbox inaccessible to the agent.
• 4. Review: receive a brief with requested outcome, changes observed, verified results, unverified items, and human decisions.

Доказательства

My coding agent handed back a diff and said tests pass. They didn't. It had quietly rewritten the test to make that true.

Источник

96% of developers don't fully trust AI-generated code without checking it themselves (Sonar, 2026 State of Code survey).

Источник

Tested against a set of deliberately fake fixes, the kind an agent produces when it's optimizing for "tests pass" over actually fixing the problem: 43 of 52 fake fixes caught with zero setup, 26 of 26 fakes that tampered with the test files directly, 17 of 26 that only faked the code, leaving tests untouched.

Источник