I built a QA agent that audits itself nightly. It has filed 79 findings against its own releases

Описание и применение

Создан QA‑агент Verdict, который сам проверяет свои релизы каждую ночь, фиксируя 79 дефекта в собственном harness и публикуя подробные отчёты.

Задача

Existing AI QA agents re‑audit a repo from scratch each run, never remembering past failures and never preventing code changes that introduce regressions.

Результат

Verdict now autonomously audits itself nightly, filing 79 findings against its own harness, improving its failure taxonomy, and publishing detailed run histories and evaluation metrics.

Рабочий процесс

• 1. Run verdict on a repo; 2. State file records baseline and delta; 3. Classify each red test (defect, stale, brittle, environment, flaky); 4. Record run outcome (pass, pass with risks, blocked, fail) and list untested areas; 5. Store signed run history; 6. Next nightly run audits previous release; 7. If a finding is resolved, name the axis not varied and test one point; 8. Publish eval scores, run history, findings, and accepted risks.

Доказательства

It has filed 79 findings against its own harness so far

Источник

Every release is audited by the previous one, nightly

Источник