Reachability Receipts: Prove an Agent Patch's Green Build Came From Tests That Touched It

Описание и применение

Формат «receipt» фиксирует, какие тесты покрывают изменённые строки, предотвращая использование флуктуационных тестов как доказательства успешного билда.

Задача

A green build can be reported even when no test actually exercised the patched lines, allowing flaky tests to be frozen and falsely used as evidence.

Результат

The receipt format explicitly records which tests touched the patched lines, preventing flaky tests from being used as evidence for a green build.

Рабочий процесс

• Derive changed line ranges with git diff -U0 and count hunks
• Collect per-test line hits using coverage.py with dynamic_context=test_function and --show-contexts
• Convert coverage JSON to JSONL format for the receipt
• Run the receipt script to intersect changed lines with test hits and check against the freeze set

Доказательства

A green build is a fact about your test suite, not about the patch.

Источник

Everything else is bookkeeping. The verdict comes from intersecting the first two, then asking the third whether that intersection is trustworthy.

Источник