Why 100% Line Coverage is Lying to You in AI-Generated Code (And How We Catch It)

Описание и применение

DeployProof — инструмент для проверки кода, генерируемого ИИ, который выявляет пустые тесты и другие уязвимости, используя дифф‑ориентированное мутационное тестирование и статические проверки, ускоряя процесс до 2–4 секунд.

Задача

AI coding assistants often generate tests that achieve 100% line coverage but miss critical edge‑case logic, leading to silent failures in production.

Результат

DeployProof reduces mutation testing time from minutes to 2–4 seconds by targeting only the diffed lines, catching hollow tests and other AI‑generated code pitfalls before they reach production.

Рабочий процесс

• 1. Install DeployProof via pip.
• 2. Run deployproof init to set up pre‑push hook and config.
• 3. Make code changes and generate tests with AI assistant.
• 4. Run deployproof check to perform diff‑scoped AST mutation testing and additional static verifications.
• 5. If any check fails, fix code/tests and re‑run until all checks pass.
• 6. Push code; pre‑push hook blocks if checks fail.
• 7. In CI/CD, run deployproof check --json for structured output.

Доказательства

DeployProof solves this with Diff‑Scoped AST Mutation: Instead of mutating the entire repository, DeployProof inspects your active git diff and targets AST mutations strictly to the lines you just wrote or modified.

Источник

This drops verification time from minutes down to 2 to 4 seconds.

Источник

DeployProof runs 5 additional static verification passes against your active diff: PyPI Dependency & Slopsquatting Defense, GhostApproval Symlink Traps, Control Flow & Error Handling, Mock‑Introduction Auditing, Credential Scanner.

Источник