Why a machine must be allowed to be wrong

说明与用途

自我改进AI的门控框架确保独立审计，防止未经验证的进步声明。

问题

Self‑improving AI claims are often untestable because agents report their own improvement without external verification, leading to unchallenged progress claims.

结果

The gated self‑improvement framework successfully promoted Gemini’s first proposal while rejecting subsequent ones that did not meet the noise margin, demonstrating that independent auditing prevents self‑justifying progress claims.

工作流

• 1. Agent generates a hypothesis and experimental plan.
• 2. Agent submits proposal to the gate. 3. Gate records a receipt and forwards proposal to an independent auditor. 4. Auditor evaluates proposal on unseen synthetic worlds, ensuring no data leakage. 5. Auditor votes; if margin exceeded, proposal is promoted to canon. 6. All receipts and audit logs are published for reproducibility.
• 7. Repeat for subsequent turns, maintaining audit trail and noise margins.

证据

In three live runs, all published, the gate promoted Gemini's first proposal and refused its next two — every time.

来源

Every verdict writes a receipt sufficient to recompute it, and an independent auditor — a different model — reads each promotion and may dissent, on the record, without veto.

来源