Can you trust a green test written by AI? I measured it.

Description and use

The False-Confidence Gap is substantial: AI-generated oracles show ~62% gap (most dangerous), LLM-judge ~41%, and AI self-healing ~11%. Branch-heavy code is especially vulnerable (grading function 99.7%, triangle classifier 93.6%). A calibrated trust metric over observable signals significantly outperforms the naive 'trust any green' baseline (ROC-AUC 0.50 vs up to ~0.95), reducing silent false-passes dramatically (self-healing 11%→0.5%, LLM-judge 40%→19%).

Problem

When AI generates test oracles, heals broken UI locators, or adjudicates pass/fail via LLM-as-judge, the dashboard stays green while the safety net quietly grows holes — weak assertions pass silently, self-healing locators repair onto wrong elements, and LLM judges wave genuine failures through as flaky. The core problem is that 'green' no longer means 'sound' for these AI-driven testing modalities.

Outcome

The False-Confidence Gap is substantial: AI-generated oracles show ~62% gap (most dangerous), LLM-judge ~41%, and AI self-healing ~11%. Branch-heavy code is especially vulnerable (grading function 99.7%, triangle classifier 93.6%). A calibrated trust metric over observable signals significantly outperforms the naive 'trust any green' baseline (ROC-AUC 0.50 vs up to ~0.95), reducing silent false-passes dramatically (self-healing 11%→0.5%, LLM-judge 40%→19%).

Workflow

• Define False-Confidence Gap = how often tests appear to pass minus how often they are actually sound.
• Use mutation testing as ground truth: inject real faults into program code and require verdicts to catch those faults.
• Run experiments across three modalities: AI-generated oracles, AI self-healing locators, LLM-as-judge adjudication.
• Repeat over 15 random seeds with held-out evaluation.
• Train a calibrated trust metric on observable runtime features.
• Compare trust metric to naive policy ('trust any green') using ROC-AUC and calibration metrics.
• Apply trust gating: keep the 70% most-trusted green verdicts, set aside the least-trusted 30%.
• Measure reduced silent false-pass rates under the trust-gated policy.

Evidence

False-Confidence Gap = how often tests appear to pass − how often they are actually sound.

An AI-written assertion can be too weak to catch a real bug — it checks the type but not the value, so the test passes even when the output is wrong.

A self-healing locator can 'repair' itself onto the wrong element and sail through, silently testing nothing.

An LLM judge can wave a genuine failure through as a flake.

AI-generated oracles: ~62% gap. This is the scary one. A huge share of 'passing' came from assertions that don't actually catch injected faults.

LLM-as-judge: ~41% gap.

AI self-healing: ~11% gap — lower, but with silent mis-heals hiding in the tail.

Branch-heavy logic like a grading function (99.7% gap) or a triangle classifier (93.6%) was far more dangerous than a simple BMI bucket (5.2%).

If you keep the 70% most-trusted green verdicts and set aside the least-trusted 30%, the silent false-pass rate drops hard: Self-healing 11% → 0.5%, LLM-judge 40% → 19%.

The naive 'trust any green' baseline can't separate sound from unsound