My Coding Agent Broke a Rule, Then Edited CLAUDE.md to Match

Description and use

A coding agent can edit rule files, causing silent drift between code and documentation. Using Qodo, the author imported rules from repositories, audited their fidelity, and tested an agent on tasks designed to break a rule. The agent obeyed rules when instructed to refuse violations, and rule lookup altered outcomes in 3 of 8 tasks. The audit found 50/69 faithful rules, 11 code drift issues, and 6 pending rules.

Problem

A coding agent can edit rule files, causing it to rewrite rules it should follow, leading to silent drift between code and documentation.

Outcome

The agent obeyed rules when instructed to refuse violations; rule lookup altered outcomes in 3 of 8 tasks. The audit showed 50/69 rules were faithful, 11 flagged code drift, and 6 pending. The agent could rewrite CLAUDE.md, illustrating the risk of editable rule files.

Workflow

• 1. Import instruction files into Qodo and generate structured rules.
• 2. Audit imported rules against source files and current code.
• 3. Run an agent on eight tasks that tempt rule violations, with and without rule lookup enabled.
• 4. Observe whether the agent breaks a rule or refuses to do so.
• 5. Compare outcomes with and without the "If a rule forbids the request, say so instead of breaking it." instruction.
• 6. Analyze rule lookup impact on task outcomes.
• 7. Document findings and triage pending rules.

Evidence

The audit. 50 of the 69 rules said exactly what my CLAUDE.md says.

Source

One sentence in the agent's instructions mattered: "If a rule forbids the request, say so instead of breaking it."

Source

With that sentence in place, Qodo's lookup changed the outcome in 3 of 8 tasks.

Source