Your CI is green. Your tests might be lying.

Описание и применение

Инструмент testguard обнаруживает шаблоны, скрывающие реальные ошибки в тестах, и позволяет использовать их как проверку в CI, предотвращая слияние PR‑ов, которые ослабляют покрытие.

Задача

A test suite can appear green while actually not exercising production code, due to patterns that mask failures.

Результат

testguard identifies 10 suspicious patterns in a faulty test file and none in a clean one, enabling teams to catch hidden bugs before they reach production.

Рабочий процесс

• Run testguard on the test directory (e.g., python -m testguard tests/).
• Review the report for suspicious patterns and their severity.
• Configure CI to fail on high‑severity findings using --fail-on high or report only with --fail-on none.
• In PR review, use diff mode (testguard --diff HEAD main) to detect added or removed test patterns before merge.

Доказательства

I shipped a bug last week that my test suite had "covered." The suite was green. The test just wasn't testing anything anymore.

Источник

testguard scans for exactly those patterns: empty-test, fake-pass, retry-loop, swallowed-assert, deleted-test, removed-assert, focused-test, no-assert, hardcoded-sleep, mock-only, skipped-test, expected-fail.

Источник

testguard reports patterns, not guilt. Treat each finding as "look here and decide", not "this code is cheating".

Источник