Why Your AI Agent Keeps Gaslighting You (And How to Fix It)

Description and use

The article discusses the gaslighting issue in AI agents, where LLMs generate plausible but incorrect outputs. The author added a verification layer that detects and corrects fabricated citations, invalid code, and other errors, reducing hallucination rates from ~18% to <3%.

Problem

AI agents often hallucinate, producing fabricated API responses, file paths, code edits, or citations that appear confident, leading users to trust incorrect outputs and blame themselves when failures occur.

Outcome

Implemented a model-agnostic, CPU-only verification layer that reduced hallucination rate from ~18% to under 3%.

Workflow

• 1. Agent generates output based on user prompt.
• 2. Verification layer receives the output immediately after generation.
• 3. Verification checks for fabricated citations, invalid code, tool call hallucinations, and coherence with the original prompt.
• 4. If any check fails, the output is corrected or flagged.
• 5. Corrected or flagged output is presented to the user.

Evidence

Result: hallucination rate dropped from ~18% to under 3%.

Source