I Gave the Same Governance Audit to One AI and an AI Swarm. They Missed the Same Bug.

Описание и применение

Эксперимент показал, что оркестровка ИИ улучшает покрытие, но не устраняет проблемы внимания и может потерять конкретные доказательства.

Задача

Determining whether a single AI or an orchestrated swarm can reliably detect governance defects across a large corpus, and understanding the trade-offs between coverage, attention, and evidence preservation.

Результат

The swarm achieved full document assignment and richer cross‑domain findings but still missed a mechanical reference bug and lost specificity in evidence, revealing that orchestration improves coverage but can dilute actionable evidence.

Рабочий процесс

• 1. Prepare a sealed benchmark of eight conditions within the governance corpus.
• 2. Mount the corpus read‑only in a Docker container.
• 3. Run Configuration A: prompt a single Claude Opus 5 to audit the corpus without prior context or orchestration.
• 4. Record findings and runtime (≈14 min).
• 5. Run Configuration B: coordinator partitions all 341 documents, assigns them to three analyst agents, and a synthesis agent consolidates reports.
• 6. Record findings and runtime (≈43 min).
• 7. Compare results against the hidden benchmark.
• 8. Analyze coverage, attention, and evidence preservation differences.
• 9. Document lessons on deterministic rule enforcement and synthesis impact.

Доказательства

The swarm did things the single agent could not.

Источник

Both missed the broken reference.

Источник