I built a QA agent that audits itself nightly. It has filed 79 findings against its own releases

Description and use

Verdict is a QA agent that audits its own releases nightly, filing 79 findings against its harness and publishing detailed run histories.

Problem

Existing AI QA agents re‑audit a repo from scratch each run, never remembering past failures and never preventing code changes that introduce regressions.

Outcome

Verdict now autonomously audits itself nightly, filing 79 findings against its own harness, improving its failure taxonomy, and publishing detailed run histories and evaluation metrics.

Workflow

• 1. Run verdict on a repo; 2. State file records baseline and delta; 3. Classify each red test (defect, stale, brittle, environment, flaky); 4. Record run outcome (pass, pass with risks, blocked, fail) and list untested areas; 5. Store signed run history; 6. Next nightly run audits previous release; 7. If a finding is resolved, name the axis not varied and test one point; 8. Publish eval scores, run history, findings, and accepted risks.

Evidence

It has filed 79 findings against its own harness so far

Source

Every release is audited by the previous one, nightly

Source