My Agent Forgot Everything Mid-Task: A Debugging Retrospective

说明与用途

LLM 代理因上下文窗口截断失去了关键决策，导致行为不一致。解决方案是记录完整对话、重放会话、模拟截断，并将过去的决策压缩为系统消息账本，以保留约束。

问题

An LLM agent lost critical decisions from its context window during implementation, causing it to ignore earlier plans and constraints.

结果

By logging the full transcript and summarizing past decisions into a system message, the agent no longer loses critical constraints when its context window is truncated, restoring consistent behavior.

工作流

• 1. Log every request and response to a JSONL file with token counts and timestamps.
• 2. Replay the entire session from the beginning to inspect context truncation.
• 3. Build a reproduction harness (simulate_truncation) to model sliding‑window behavior.
• 4. Verify which key facts survive or are lost when the token budget is exceeded.
• 5. Replace simple truncation with compress_and_keep: summarize past turns into a system message and keep recent turns verbatim.
• 6. Test the new approach to confirm constraints persist.
• 7. Log and audit the summaries for traceability.

证据

The output looked fine in isolation, which made the failure harder to spot, and I spent the first hour assuming the model was simply bad.

来源

The fix was to compress decisions instead of discarding them: every few turns, a summarizer reads the full transcript and writes a short decision ledger that stays pinned in the context as a system message.

来源