Is it done, or does it just look done? A ladder of evidence for AI-agent work

说明与用途

确定 AI 代理工作是否真正完成，需要超越自我报告的证据。Agateon 设计了从自我报告到独立判断的五级证据阶梯，并提供了审核每个门控的清单。

问题

Determining when an AI agent’s work is truly complete is hard because agents can omit, fabricate, or confidently misreport their actions, leading to unsafe or unreliable automation.

结果

By shifting gate evidence from agent self‑reports to external sources—artifacts, tool verdicts, git history, and independent reviewers—Agateon reduced silent failures and increased confidence that AI‑agent work is truly complete.

工作流

• 1. Identify each gate that marks a phase as done.
• 2. Document the exact evidence it checks (file, field, command, exit code).
• 3. Trace the author of that evidence (agent, tool, or world).
• 4. Verify the gate cannot be bypassed by omission (empty or missing evidence).
• 5. Verify the gate cannot be bypassed by fabrication (plausible but false evidence).
• 6. Promote gates to higher evidence rungs where possible (e.g., use git history or tool verdicts).
• 7. Add independent judgment gates for non‑automatable decisions.
• 8. Iterate and audit gates using the five‑question checklist.

证据

The safety net from the postmortem was a gate that read the retry counter; the question this post answers is how to tell a gate that reads real evidence from a gate that reads the agent's own story.

来源

The ladder we use, weakest to strongest: 1 Self‑report, 2 Produced artifacts, 3 Tool verdicts, 4 Immutable history, 5 Independent judgment.

来源