I Tracked Every AI Hallucination for a Week — The Numbers Were Worse Than I Thought
说明与用途
作者发现约18%的AI输出错误。他构建了一个验证层,自动检查引用、代码、泄漏和参数,节省令牌和调试时间。
问题
AI agents frequently produce hallucinated outputs, wasting token budget and causing debugging effort.
结果
The verification layer reduced hallucinations, saving nearly 20% of token budget and decreasing manual debugging effort.
工作流
- Run AI model to generate output
- Pass output to verification layer
- Citation checker validates references
- Code validator checks syntax and logic
- Safety leak detector identifies leaked prompts
- Argument verifier checks tool call parameters
- Coherence scorer compares output to prompt
- If all checks pass, output is accepted; otherwise, it is flagged for review
证据
Out of 200 outputs across Claude, GPT, and DeepSeek: 36 were confidently wrong (18%)
All runs in under 100ms on CPU. Model-agnostic. Free.
- 记录日期: 2026-09-06T22:07:18.951Z