Why Our CLI Deadlock Detector for Multi-Agent LLMs Timed Out: A Post-Mortem on Naive Jaccard Heuristics

Description and use

The CLI deadlock detector for multi‑agent LLMs timed out during high‑concurrency tests because of O(N²) similarity calculations and naive tokenization, failing to meet the 10‑second SLA.

Problem

The Asynchronous Semantic Deadlock Checker failed to complete within the required 10‑second SLA during high‑concurrency QA stress tests involving more than 20 active agents.

Outcome

The tool collapsed under high concurrency, failing to meet the 10‑second SLA, highlighting the inefficiency of naive tokenization and pairwise similarity for large agent sets.

Workflow

• 1. Read JSON array of agent logs from stdin
• 2. Tokenize each agent’s output using naive whitespace splitting
• 3. Build a pairwise Jaccard similarity matrix (O(N²))
• 4. Flag pairs with similarity below 0.15 as divergent
• 5. Identify the agent with the highest outlier distance
• 6. Emit a JSON diagnostic payload to stdout

Evidence

The goal was simple: consume execution traces, calculate semantic alignment across all collaborating agents, and trigger recovery interventions within a strict 10-second SLA.

Source

During high-concurrency QA stress tests involving 20+ active agents, the tool collapsed with an unhandled timeout.

Source