My tests were describing the code, not checking it

说明与用途

AI生成的测试可能通过，但未检测到真实边缘情况，导致错误的四舍五入。手动变异测试发现覆盖缺口，随后修正了四舍五入逻辑并移除了有问题的功能，并纠正了发布流程。

问题

Automated tests written by an AI agent can pass while still failing to validate real-world edge cases, leading to incorrect production behavior.

结果

The project uncovered that AI-generated tests can be self-consistent yet blind to real-world edge cases. Manual mutation testing exposed missing coverage, leading to corrected rounding logic and removal of a problematic feature. The library was updated and released correctly.

工作流

• 1. Generate library and tests with Claude Code
• 2. Observe green CI and passing tests
• 3. Notice incorrect rounding on a UK invoice
• 4. Perform mutation testing by hand: mutate rounding calls, run tests, record failures
• 5. Identify missing coverage of rounding helper usage
• 6. Change rounding policy to MidpointRounding.AwayFromZero
• 7. Add tests for rounding helper and invoice calculations
• 8. Detect per-line tax mode issue via design questions
• 9. Remove per-line tax mode and add constructor guard
• 10. Fix release pipeline to bump version correctly
• 11. Publish updated library to NuGet

证据

I found it because I got suspicious of my own green checkmark.

来源

The library rounds half-up, MidpointRounding.AwayFromZero, because that is what tax authorities expect.

来源

Under de-DE, 100m.ToString() gives you "100". Same as invariant.

来源