Loading page...Загружаем страницу...正在加载页面...
AtlasRepo

Two Readers Broke My Checks. Not One of My Numbers Moved.

说明与用途

开发者发现并修复了变异测试工具中的隐藏错误,使用字节大小保持的“金丝雀”与负控制提升了覆盖率准确性至95.7%。

问题

The evaluation harness for measuring test suite mutation coverage contained hidden bugs that allowed stale bytecode to be used, leading to false positives and unreliable metrics.

结果

The harness now accurately reports mutation coverage, with a 95.7% kill rate, and includes safeguards against stale bytecode and silent failures.

工作流

  • 1. Identify mutation bug: stale .pyc usage due to CPython mtime granularity.
  • 2. Add byte‑size‑preserving canary to detect stale bytecode.
  • 3. Verify no .pyc files are copied into temp directories.
  • 4. Add negative control suite of vacuous tests expecting near‑zero score.
  • 5. Detect and fix proxy test that relied on pyc count.
  • 6. Re‑write test to assert observed outcome rather than proxy.
  • 7. Document load‑bearing safeguards and environment variable handling.
  • 8. Re‑run full mutation test suite and confirm kill rate of 95.7%.
  • 9. Publish harness and results for community scrutiny.

证据

I added a byte-size-preserving canary alongside the original. Eight of twelve targets are eligible and all pass.

来源

It killed 7 of 51 mutants. 13.7%.

来源

  • 记录日期: 2026-09-17T00:30:20.738Z