Debugging AI Agent Guardrails: When Safety Rails Lie

Description and use

The author encountered an AI agent in NoCoder falsely reporting missing files despite successful writes. The issue stemmed from a guardrail that incorrectly assumed a file existed based on previous write actions, ignoring its actual deletion. The fix involved making the guardrail verify the file's existence on disk rather than relying on write history. This highlights the importance of accurate feedback from guardrails for AI agents.

Problem

An AI agent in NoCoder appeared incompetent, claiming files were not persisting despite success reports, and defensively rewriting files sequentially to avoid a non-existent race condition.

Outcome

The issue was resolved by updating the loop guard to verify file existence rather than assuming persistence based on previous write actions. This prevented the agent from receiving false feedback about missing files.

Workflow

• Observe agent behavior: Agent claims files are missing despite success reports.
• Formulate initial hypothesis: Suspect approval gate is queueing writes.
• Verify hypothesis: Read the call site code for the approval gate.
• Refute hypothesis: Confirm the gate is blocking, not queueing; the agent waits for approval before writing.
• Identify actual cause: Discover the loop guard assumes a file exists if it was written previously, ignoring intermediate deletions.
• Implement fix: Modify the guard to check the filesystem for the file's existence before skipping a rewrite.
• Validate fix: Confirm that the agent no longer falsely infers workspace corruption.

Evidence

If the agent issues a second write to a path it already wrote, the guard assumes the model is spinning, skips the write, and feeds back: 'You already wrote X. Do NOT rewrite it.'

Source

Make the guard check instead of assume. It now probes the workspace for the file and only skips the rewrite when the file is genuinely still present.

Source

Guardrails are part of the model's world. We spend a lot of effort on prompts and tools and comparatively little on what our safety rails say when they fire.

Source