Bugs Are Innocent Until Reproduced: Building Verdict, an Evidence-First Agent Harness

说明与用途

Verdict 将错误转化为可重现的记录，利用证据和严格阈值来降低调试间歇性错误的不确定性。

问题

Flaky bug reports often end as "cannot reproduce" or patches that cannot be proven fixed, leaving maintainers uncertain about the true state of a defect.

结果

Verdict provides a reproducible, evidence-based record of flaky bugs, enabling maintainers to verify fixes with deterministic thresholds and reducing uncertainty in bug triage.

工作流

• Create a GitHub issue for the bug.
• Configure Verdict with approved commands, knobs, and budgets.
• Hunter searches the condition matrix and records all runs.
• Surgeon narrows the reproduced condition to the smallest suspect range.
• Insurance converts the reproduction into a regression plan and drafts a PR.
• Maintainer reviews the evidence and approves the PR.
• Verdict publishes the reproducible record and evidence hash.
• Reproduction can be recomputed with pnpm --filter @verdict/agent verify:runtime-evidence.
• CI runs the same verification steps to ensure consistency.

证据

Verdict turns a GitHub issue into a bounded investigation. It runs an approved command repeatedly under approved conditions, keeps every observation and refuses to claim a reproduction unless the evidence crosses a deterministic threshold.

来源

The verifier returns: {"verdict":"REPRODUCED",...}

来源