When AI writes the fix and the test together, is PASS enough?

Описание и применение

Timewitness позволяет проверить, действительно ли тест обнаруживает исправленный баг, сравнивая его выполнение до и после исправления, без необходимости вручную откатывать изменения.

Задача

When an AI coding assistant writes a fix and a regression test in the same change, a passing test does not guarantee the test would have failed before the fix, making it hard to verify that the test actually exercises the bug.

Результат

Timewitness reliably confirms that a test distinguishes the pre‑fix and post‑fix states, with zero false PROVEN results in the reported runs.

Рабочий процесс

• 1. Run node timewitness.cjs arm to snapshot the current working tree as the "before" state.
• 2. Edit code and add or modify a regression test as usual.
• 3. Run node timewitness.cjs prove --test <test-file> to execute the test against both the before and after copies.
• 4. Review the output: PROVEN indicates the test failed before and passed after; NOT_PROVEN indicates no FAIL‑then‑PASS pattern; INCONCLUSIVE indicates unreliable comparison.

Доказательства

TIMEWITNESS: PROVEN Before: FAIL 3/3 After : PASS 3/3

TIMEWITNESS: NOT_PROVEN Before: PASS 2/2 After : PASS 2/2 Reason: BEFORE_ALSO_PASSES

Across those two runs, false PROVEN (a case where it should not have said PROVEN but did) occurred 0 times.