No agent grades its own homework

Description and use

When an LLM agent reviews its own code it echoes its mistakes. Agateon solves this by separating roles: an orchestrator dispatches work, a fresh subagent implements it, and an independent judge verifies the artifact in a new context using a machine‑checked exit code. This eliminates the echo and improves reliability.

Problem

When an LLM agent reviews its own output it echoes its own misunderstandings, missing bugs and misinterpretations.

Outcome

The new structure prevented two bad commits that existed only in logs, caught a writer agent’s fabricated anecdote, and ensured that verification was independent of author context, improving reliability.

Workflow

• 1. Orchestrator reads state and dispatches a phase to a fresh subagent with only the brief and protocol files.
• 2. Subagent implements the task and writes output to a read‑only artifact.
• 3. Judge agent is spawned in a brand‑new session, receives only the artifact and spec, and runs automated tests, producing an exit code.
• 4. Acceptance gate checks the exit code; if it passes, the artifact is committed; if not, the orchestrator retries or aborts.
• 5. All retries and outcomes are logged with round number, failure mode, and prompt changes for auditability.

Evidence

We ran our whole engineering workflow this way, and it has caught things the author‑agent could not have caught: two bad commits that exist only in a log and never in git, an implementation sent back twice by a judge that had never seen our reasoning, and — a few days ago — a blog draft where a writer agent invented a personal anecdote to sound more human, which a fresh‑context reviewer flagged while the writer never noticed.

Source

The orchestrator never touches the artifact. It reads state, dispatches phases, and runs gate commands — read‑only verification, never file edits.

Source