Why a machine must be allowed to be wrong

Описание и применение

Система «ворот» обеспечивает независимый аудит самосовершенствования ИИ, предотвращая необоснованные заявления о прогрессе.

Задача

Self‑improving AI claims are often untestable because agents report their own improvement without external verification, leading to unchallenged progress claims.

Результат

The gated self‑improvement framework successfully promoted Gemini’s first proposal while rejecting subsequent ones that did not meet the noise margin, demonstrating that independent auditing prevents self‑justifying progress claims.

Рабочий процесс

• 1. Agent generates a hypothesis and experimental plan.
• 2. Agent submits proposal to the gate. 3. Gate records a receipt and forwards proposal to an independent auditor. 4. Auditor evaluates proposal on unseen synthetic worlds, ensuring no data leakage. 5. Auditor votes; if margin exceeded, proposal is promoted to canon. 6. All receipts and audit logs are published for reproducibility.
• 7. Repeat for subsequent turns, maintaining audit trail and noise margins.

Доказательства

In three live runs, all published, the gate promoted Gemini's first proposal and refused its next two — every time.

Источник

Every verdict writes a receipt sufficient to recompute it, and an independent auditor — a different model — reads each promotion and may dissent, on the record, without veto.

Источник