AI-Generated Tests: How Many Actually Test Different Behaviors?

Описание и применение

Автоматически генерируемые тесты часто дублируют друг друга, но не проверяют разные поведения. Инструмент vitest‑dedup 0.5.0 различает идентичные, почти дублирующие и действительно разные тесты, позволяя повысить качество покрытия.

Задача

Automated test generation can produce many passing tests that appear to increase coverage, but many of those tests are duplicates or near‑duplicates that do not test distinct behaviors.

Результат

vitest‑dedup 0.5.0 correctly distinguishes between identical tests, near‑duplicates, and tests that exercise different behaviors, whereas a naive token‑based detector misclassifies many cases.

Рабочий процесс

• Run coding agent to generate tests
• Classify suspicious pairs manually
• Run three detectors: naive token‑based, vitest‑dedup 0.4.0, vitest‑dedup 0.5.0
• Compare results and analyze verdicts

Доказательства

A coding agent can write a hundred passing tests before you finish your coffee. The test count goes up, the coverage report turns green, and the pull request looks thorough.

Источник

vitest-dedup encodes this split in its output. Every finding carries a verdict: redundant, consolidatable, or a review prompt.

Источник

The naive detector reports them as exact duplicates. The bodies are the same text, but clock is a different binding in each one, and that binding is the test.

Источник

vitest-dedup 0.5.0 reports neither pair. Captured variables participate in the comparison at the place they are used, and they keep track of which enclosing variable they came from.

Источник

0.5.0 models local classes: fields, methods, constructors, private names and what each instance captures. Consistent renaming stays comparable, so the verdict becomes a proven duplicate-test.

Источник