When AI writes the fix and the test together, is PASS enough?

说明与用途

Timewitness 通过在修复前后分别运行测试，验证测试是否真正检测到错误，避免手动回滚操作。

问题

When an AI coding assistant writes a fix and a regression test in the same change, a passing test does not guarantee the test would have failed before the fix, making it hard to verify that the test actually exercises the bug.

结果

Timewitness reliably confirms that a test distinguishes the pre‑fix and post‑fix states, with zero false PROVEN results in the reported runs.

工作流

• 1. Run node timewitness.cjs arm to snapshot the current working tree as the "before" state.
• 2. Edit code and add or modify a regression test as usual.
• 3. Run node timewitness.cjs prove --test <test-file> to execute the test against both the before and after copies.
• 4. Review the output: PROVEN indicates the test failed before and passed after; NOT_PROVEN indicates no FAIL‑then‑PASS pattern; INCONCLUSIVE indicates unreliable comparison.

证据

TIMEWITNESS: PROVEN Before: FAIL 3/3 After : PASS 3/3

TIMEWITNESS: NOT_PROVEN Before: PASS 2/2 After : PASS 2/2 Reason: BEFORE_ALSO_PASSES

Across those two runs, false PROVEN (a case where it should not have said PROVEN but did) occurred 0 times.