Why Our CLI Deadlock Detector for Multi-Agent LLMs Timed Out: A Post-Mortem on Naive Jaccard Heuristics

Описание и применение

CLI‑инструмент для обнаружения семантических дедлоков в многокомпонентных LLM‑оркестрациях не успел завершить работу за 10 секунд при нагрузке более 20 агентов из‑за O(N²) расчётов и простого токенизатора.

Задача

The Asynchronous Semantic Deadlock Checker failed to complete within the required 10‑second SLA during high‑concurrency QA stress tests involving more than 20 active agents.

Результат

The tool collapsed under high concurrency, failing to meet the 10‑second SLA, highlighting the inefficiency of naive tokenization and pairwise similarity for large agent sets.

Рабочий процесс

• 1. Read JSON array of agent logs from stdin
• 2. Tokenize each agent’s output using naive whitespace splitting
• 3. Build a pairwise Jaccard similarity matrix (O(N²))
• 4. Flag pairs with similarity below 0.15 as divergent
• 5. Identify the agent with the highest outlier distance
• 6. Emit a JSON diagnostic payload to stdout

Доказательства

The goal was simple: consume execution traces, calculate semantic alignment across all collaborating agents, and trigger recovery interventions within a strict 10-second SLA.

Источник

During high-concurrency QA stress tests involving 20+ active agents, the tool collapsed with an unhandled timeout.

Источник