Freeze the Counterexample, Not the Test File

Описание и применение

Гарантирует неизменность фикстур, предотвращая подмену тестов агентами.

Задача

Agent patches often sabotage tests by rewriting golden files, wrapping assertions, or inflating timeouts, making it hard to detect genuine failures.

Результат

Tests now run against immutable, verified fixtures, eliminating hidden failures caused by agent patches.

Рабочий процесс

• 1. Store fixture bytes in oracles/objects/<sha256> and map names in oracles/manifest.json.
• 2. Run tools/verify_manifest.py before tests to ensure all digests match stored objects and no drift exists.
• 3. Use property suite to load fixtures by digest, not path.
• 4. Record any new counterexamples in the flake ledger and add them as objects.
• 5. Fail CI if manifest names change or objects drift, but allow orphans as warnings.

Доказательства

Phase-1 CI should fail closed if a name moved or a file drifted.

Источник