What makes an agent expensive is not what it does, it is what it remembers

说明与用途

对话代理的高成本主要由重复读取上下文导致，而非实际工作。日志分析显示94％的令牌用于重新读取，通过过滤上下文可将成本降低约三分之一。

问题

High costs of conversational AI agents are driven by repeated context re‑reading rather than the actual work performed.

结果

Re‑reading accounted for ~94% of token usage; filtering reduced per‑turn context by an order of magnitude, cutting overall spend by roughly a third.

工作流

• 1. Collect .jsonl session logs with usage blocks.
• 2. Run script to calculate token cost per session and per call.
• 3. Identify re‑reading proportion and context growth per turn.
• 4. Attribute tool results to calls and compute carry cost.
• 5. Rank calls by carry to find high‑impact items.
• 6. Implement filtered readers, new sessions per topic, and output‑to‑file strategy.
• 7. Re‑measure to confirm cost reduction.

证据

Out of 2.36 billion tokens, what the model wrote (the code, the answers, the analysis) came to 19 million. Less than one percent.

来源

54 % is reading whole files. And inside that, the biggest surprise: a good part of it was not code files but the agent's own memory files.

来源