Safety Rules Don’t Survive the Context Window: Quantifying and Fixing Agent Compliance Decay

Description and use

<h1> Safety Rules Don’t Survive the Context Window: Quantifying and Fixing Agent Compliance Decay </h1> <h2> Injected Safety Rules Are Ignored as Context Windows Fill </h2> <p>Anyone deploying LLM agents with hardcoded safety rules has seen this: as dialogs grow, initial guardrails disappear. The cause is structural. Once the system prompt—“Never give medical advice,” “Decline illegal requests,” etc.—is pushed out of the model's context window, it vanishes from generation. Agents get riskier, and compliance disintegrates, regardless of careful prompt engineering.</p> <p>The problem appears in

Problem

<h1> Safety Rules Don’t Survive the Context Window: Quantifying and Fixing Agent Compliance Decay </h1> <h2> Injected Safety Rules Are Ignored as Context Windows Fill </h2> <p>Anyone deploying LLM agents with hardcoded safety rules has seen this: as dialogs grow, initial guardrails disappear. The cause is structural. Once the system prompt—“Never give medical advice,” “Decline illegal requests,” etc.—is pushed out of the model's context window, it vanishes from generation. Agents get riskier, and compliance disintegrates, regardless of careful prompt engineering.</p> <p>The problem appears in

Outcome

<h1> Safety Rules Don’t Survive the Context Window: Quantifying and Fixing Agent Compliance Decay </h1> <h2> Injected Safety Rules Are Ignored as Context Windows Fill </h2> <p>Anyone deploying LLM agents with hardcoded safety rules has seen this: as dialogs grow, initial guardrails disappear. The cause is structural. Once the system prompt—“Never give medical advice,” “Decline illegal requests,” etc.—is pushed out of the model's context window, it vanishes from generation. Agents get riskier, and compliance disintegrates, regardless of careful prompt engineering.</p> <p>The problem appears in

Evidence

Safety Rules Don’t Survive the Context Window: Quantifying and Fixing Agent Compliance Decay

Source