My Agent Forgot Everything Mid-Task: A Debugging Retrospective

Описание и применение

Агент LLM потерял ключевые решения из окна контекста, что привело к несоответствию планов. Решение: логировать всю переписку, воспроизводить сессии, моделировать обрезку, а затем использовать резюмирующий леджер, сохраняющий решения как системное сообщение.

Задача

An LLM agent lost critical decisions from its context window during implementation, causing it to ignore earlier plans and constraints.

Результат

By logging the full transcript and summarizing past decisions into a system message, the agent no longer loses critical constraints when its context window is truncated, restoring consistent behavior.

Рабочий процесс

• 1. Log every request and response to a JSONL file with token counts and timestamps.
• 2. Replay the entire session from the beginning to inspect context truncation.
• 3. Build a reproduction harness (simulate_truncation) to model sliding‑window behavior.
• 4. Verify which key facts survive or are lost when the token budget is exceeded.
• 5. Replace simple truncation with compress_and_keep: summarize past turns into a system message and keep recent turns verbatim.
• 6. Test the new approach to confirm constraints persist.
• 7. Log and audit the summaries for traceability.

Доказательства

The output looked fine in isolation, which made the failure harder to spot, and I spent the first hour assuming the model was simply bad.

Источник

The fix was to compress decisions instead of discarding them: every few turns, a summarizer reads the full transcript and writes a short decision ledger that stays pinned in the context as a system message.

Источник