Six Checks to Run Before You Trust a Number Your Own Harness Produced
说明与用途
使用六步检查清单发现十个错误,提升测量工具的可靠性。
问题
Measurement harnesses can silently produce misleading results due to bugs that are not caught by code review.
结果
The checklist uncovered ten bugs, improved harness reliability, and prevented biased measurement results.
工作流
- 1. Replace the target module with unparseable code and assert the harness fails (canary).
- 2. Run the scoring function three times and require byte‑identical output (determinism gate).
- 3. Create a vacuous test suite that should score near zero and assert it does (negative control).
- 4. Assert the actual property of interest rather than a proxy signal (property check).
- 5. Write a prediction of expected outcome before running diagnostics (prediction habit).
- 6. Install and run the harness in a fresh environment on an unrelated repository (external validation).
证据
Replace the thing under test with something unparseable and assert that your harness notices.
Run the same scoring three times, serially, and require byte-identical output.
Build one case where the score should be near zero.
- 记录日期: 2026-09-19T07:00:27.308Z