Five detectors, one bad merge: why our LLM corruption guard flagged 43% of healthy output

Описание и применение

Разработан в‑стримовый детектор для самохостинга 300‑млрд параметрической модели, который мгновенно прерывает генерацию при обнаружении «коррупции» текста, снижая ложные срабатывания до 0 % и экономя GPU‑ресурсы.

Задача

A self‑hosted 300B reasoning model intermittently produces corrupted text (e.g., fabricated news, code, or nonsensical tables) while streaming to users, causing user frustration and wasted GPU time.

Результат

The detector achieved 100% true‑positive rate and reduced false‑positive rate from 42.9% to 0.0% after per‑detector fixes, preventing corrupted text from leaking while adding only ~1 s latency.

Рабочий процесс

• 1. Buffer first 350 characters and evaluate for corruption.
• 2. If clean, flush buffer and start live stream with rechecks every 400 characters.
• 3. Run five lightweight detectors (rules, n‑gram surprise, Count‑Min Sketch, Rolling SimHash, character entropy) on each character.
• 4. Fuse detector scores using strongest‑signal dominance + corroboration bonus.
• 5. Abort generation if fused score exceeds calibrated threshold τ (set via split‑conformal calibration).
• 6. Log aborts and optionally audit historical data for missed corruptions.

Доказательства

We measured a 13,200-character table echo that took 269 seconds to generate to completion.

Источник

FPR dropped from 0.429 to 0.024 after changing fusion rule.

Источник

Zero false alarms on 119 historical messages; two real corruptions detected.

Источник