The Free Model Wrote Tests That Passed. They Still Missed a Bug

说明与用途

LLM 生成的测试可能通过，但会遗漏关键边界情况。通过添加覆盖率检查发现了端口解析函数中的错误，修复后实现了 100% 分支覆盖率。

问题

Model‑generated unit tests can pass while still missing critical edge cases, leading to undetected bugs.

结果

Coverage instrumentation uncovered a critical edge case missed by LLM‑generated tests, leading to a bug fix and 100% branch coverage.

工作流

• Generate tests with a free LLM (MonkeyCode).
• Compile and run the tests with a plain build.
• Compile and run the same tests with coverage instrumentation (gcov).
• Review the coverage report to identify untested branches.
• Add missing tests for uncovered branches.
• Re‑run tests and coverage to confirm all branches are exercised.
• Apply fixes to the code and verify full coverage.

证据

The coverage build uses the same test binary. No extra test code is needed. The only cost is a recompile and a few seconds of runtime.

来源

The model’s tests never called parse_port with an empty string.

来源