I Gave the Same Governance Audit to One AI and an AI Swarm. They Missed the Same Bug.

说明与用途

实验表明，AI编排提高了覆盖率，但未能消除注意力问题，且可能丢失具体证据。

问题

Determining whether a single AI or an orchestrated swarm can reliably detect governance defects across a large corpus, and understanding the trade-offs between coverage, attention, and evidence preservation.

结果

The swarm achieved full document assignment and richer cross‑domain findings but still missed a mechanical reference bug and lost specificity in evidence, revealing that orchestration improves coverage but can dilute actionable evidence.

工作流

• 1. Prepare a sealed benchmark of eight conditions within the governance corpus.
• 2. Mount the corpus read‑only in a Docker container.
• 3. Run Configuration A: prompt a single Claude Opus 5 to audit the corpus without prior context or orchestration.
• 4. Record findings and runtime (≈14 min).
• 5. Run Configuration B: coordinator partitions all 341 documents, assigns them to three analyst agents, and a synthesis agent consolidates reports.
• 6. Record findings and runtime (≈43 min).
• 7. Compare results against the hidden benchmark.
• 8. Analyze coverage, attention, and evidence preservation differences.
• 9. Document lessons on deterministic rule enforcement and synthesis impact.

证据

The swarm did things the single agent could not.

来源

Both missed the broken reference.

来源