Is it done, or does it just look done? A ladder of evidence for AI-agent work

Description and use

Determining when an AI agent’s work is truly complete requires evidence beyond self‑report. Agateon built a five‑rung evidence ladder—from self‑report to independent judgment—and a checklist to audit each gate.

Problem

Determining when an AI agent’s work is truly complete is hard because agents can omit, fabricate, or confidently misreport their actions, leading to unsafe or unreliable automation.

Outcome

By shifting gate evidence from agent self‑reports to external sources—artifacts, tool verdicts, git history, and independent reviewers—Agateon reduced silent failures and increased confidence that AI‑agent work is truly complete.

Workflow

• 1. Identify each gate that marks a phase as done.
• 2. Document the exact evidence it checks (file, field, command, exit code).
• 3. Trace the author of that evidence (agent, tool, or world).
• 4. Verify the gate cannot be bypassed by omission (empty or missing evidence).
• 5. Verify the gate cannot be bypassed by fabrication (plausible but false evidence).
• 6. Promote gates to higher evidence rungs where possible (e.g., use git history or tool verdicts).
• 7. Add independent judgment gates for non‑automatable decisions.
• 8. Iterate and audit gates using the five‑question checklist.

Evidence

The safety net from the postmortem was a gate that read the retry counter; the question this post answers is how to tell a gate that reads real evidence from a gate that reads the agent's own story.

Source

The ladder we use, weakest to strongest: 1 Self‑report, 2 Produced artifacts, 3 Tool verdicts, 4 Immutable history, 5 Independent judgment.

Source