Bugs Are Innocent Until Reproduced: Building Verdict, an Evidence-First Agent Harness

Описание и применение

Verdict превращает проблему в GitHub в воспроизводимое расследование, фиксируя каждую попытку и выдавая доказательства, которые можно проверить независимо.

Задача

Flaky bug reports often end as "cannot reproduce" or patches that cannot be proven fixed, leading to uncertainty and wasted effort.

Результат

Verdict produced a reproducible record for TrueForge issue #417, demonstrating 10/10 failures under a stalled endpoint and 0/10 under a responsive endpoint, with a verifiable hash and a draft PR awaiting maintainer approval.

Рабочий процесс

• Create a GitHub issue for the bug
• Configure Verdict with approved commands, knobs, and budgets
• Hunter searches the condition matrix and records all runs
• Surgeon narrows the reproduced condition to the smallest suspect range
• Insurance converts the reproduction into a regression plan and draft PR
• Maintainer reviews the evidence and approves publication
• Verifier re-runs the evidence to confirm reproducibility

Доказательства

Verdict turns a GitHub issue into a bounded investigation. It runs an approved command repeatedly under approved conditions, keeps every observation and refuses to claim a reproduction unless the evidence crosses a deterministic threshold.

Источник

The verifier returns: { "verdict": "REPRODUCED", "stalledRuns": 10, "responsiveControls": 10, ... }

Источник