AI-Generated Tests: How Many Actually Test Different Behaviors?

说明与用途

AI 生成的测试往往重复但不测试不同行为。vitest‑dedup 0.5.0 区分相同、近似重复和真正不同的测试，提升覆盖质量。

问题

Automated test generation can produce many passing tests that appear to increase coverage, but many of those tests are duplicates or near‑duplicates that do not test distinct behaviors.

结果

vitest‑dedup 0.5.0 correctly distinguishes between identical tests, near‑duplicates, and tests that exercise different behaviors, whereas a naive token‑based detector misclassifies many cases.

工作流

• Run coding agent to generate tests
• Classify suspicious pairs manually
• Run three detectors: naive token‑based, vitest‑dedup 0.4.0, vitest‑dedup 0.5.0
• Compare results and analyze verdicts

证据

A coding agent can write a hundred passing tests before you finish your coffee. The test count goes up, the coverage report turns green, and the pull request looks thorough.

来源

vitest-dedup encodes this split in its output. Every finding carries a verdict: redundant, consolidatable, or a review prompt.

来源

The naive detector reports them as exact duplicates. The bodies are the same text, but clock is a different binding in each one, and that binding is the test.

来源

vitest-dedup 0.5.0 reports neither pair. Captured variables participate in the comparison at the place they are used, and they keep track of which enclosing variable they came from.

来源

0.5.0 models local classes: fields, methods, constructors, private names and what each instance captures. Consistent renaming stays comparable, so the verdict becomes a proven duplicate-test.

来源