Loading page...Загружаем страницу...正在加载页面...
AtlasRepo

I Tracked Every AI Hallucination for a Week — The Numbers Were Worse Than I Thought

Описание и применение

Автор провёл эксперимент, выявив, что около 18 % выводов ИИ содержат ошибки. Он создал слой проверки, который автоматически проверяет цитаты, код, утечки и аргументы, экономя токены и время отладки.

Задача

AI agents frequently produce hallucinated outputs, wasting token budget and causing debugging effort.

Результат

The verification layer reduced hallucinations, saving nearly 20% of token budget and decreasing manual debugging effort.

Рабочий процесс

  • Run AI model to generate output
  • Pass output to verification layer
  • Citation checker validates references
  • Code validator checks syntax and logic
  • Safety leak detector identifies leaked prompts
  • Argument verifier checks tool call parameters
  • Coherence scorer compares output to prompt
  • If all checks pass, output is accepted; otherwise, it is flagged for review

Доказательства

Out of 200 outputs across Claude, GPT, and DeepSeek: 36 were confidently wrong (18%)

Источник

All runs in under 100ms on CPU. Model-agnostic. Free.

Источник

  • Дата записи: 2026-09-06T22:07:18.951Z