Five detectors, one bad merge: why our LLM corruption guard flagged 43% of healthy output

说明与用途

为自托管300B LLM实现了实时腐败检测器，在约350字符内中止生成，消除错误输出并节省GPU资源，误报率降至0%。

问题

A self‑hosted 300B reasoning model intermittently produces corrupted text (e.g., fabricated news, code, or nonsensical tables) while streaming to users, causing user frustration and wasted GPU time.

结果

The detector achieved 100% true‑positive rate and reduced false‑positive rate from 42.9% to 0.0% after per‑detector fixes, preventing corrupted text from leaking while adding only ~1 s latency.

工作流

• 1. Buffer first 350 characters and evaluate for corruption.
• 2. If clean, flush buffer and start live stream with rechecks every 400 characters.
• 3. Run five lightweight detectors (rules, n‑gram surprise, Count‑Min Sketch, Rolling SimHash, character entropy) on each character.
• 4. Fuse detector scores using strongest‑signal dominance + corroboration bonus.
• 5. Abort generation if fused score exceeds calibrated threshold τ (set via split‑conformal calibration).
• 6. Log aborts and optionally audit historical data for missed corruptions.

证据

We measured a 13,200-character table echo that took 269 seconds to generate to completion.

来源

FPR dropped from 0.429 to 0.024 after changing fusion rule.

来源

Zero false alarms on 119 historical messages; two real corruptions detected.

来源