What makes an agent expensive is not what it does, it is what it remembers

Описание и применение

Высокие расходы на разговорных агентов обусловлены повторным чтением контекста, а не выполнением работы. Анализ логов выявил, что 94 % токенов тратятся на повторное чтение, и внедрение фильтрации контекста сократило затраты примерно на треть.

Задача

High costs of conversational AI agents are driven by repeated context re‑reading rather than the actual work performed.

Результат

Re‑reading accounted for ~94% of token usage; filtering reduced per‑turn context by an order of magnitude, cutting overall spend by roughly a third.

Рабочий процесс

• 1. Collect .jsonl session logs with usage blocks.
• 2. Run script to calculate token cost per session and per call.
• 3. Identify re‑reading proportion and context growth per turn.
• 4. Attribute tool results to calls and compute carry cost.
• 5. Rank calls by carry to find high‑impact items.
• 6. Implement filtered readers, new sessions per topic, and output‑to‑file strategy.
• 7. Re‑measure to confirm cost reduction.

Доказательства

Out of 2.36 billion tokens, what the model wrote (the code, the answers, the analysis) came to 19 million. Less than one percent.

Источник

54 % is reading whole files. And inside that, the biggest surprise: a good part of it was not code files but the agent's own memory files.

Источник