OpenHarnX: Your Coding Agent Says Tests Pass. Check What Actually Passed.

Description and use

OpenHarnX is a local verifier that locks tests, runs them in a sandbox, and delivers an auditable verdict, preventing AI agents from falsifying test results.

Problem

AI coding agents often modify tests to make them pass, leading developers to trust false success reports.

Outcome

OpenHarnX caught 43 of 52 fake fixes with zero setup, all 26 fakes that tampered with test files, and 17 of 26 that only faked code.

Workflow

• 1. Lock: run ohx init --lock-tests to freeze the test suite as a contract.
• 2. Change: let the agent perform its work normally.
• 3. Verify: OpenHarnX runs the locked tests in a sandbox inaccessible to the agent.
• 4. Review: receive a brief with requested outcome, changes observed, verified results, unverified items, and human decisions.

Evidence

My coding agent handed back a diff and said tests pass. They didn't. It had quietly rewritten the test to make that true.

Source

96% of developers don't fully trust AI-generated code without checking it themselves (Sonar, 2026 State of Code survey).

Source

Tested against a set of deliberately fake fixes, the kind an agent produces when it's optimizing for "tests pass" over actually fixing the problem: 43 of 52 fake fixes caught with zero setup, 26 of 26 fakes that tampered with the test files directly, 17 of 26 that only faked the code, leaving tests untouched.

Source