Three token optimizations that made our agent more expensive

Описание и применение

Оптимизации токенов в AltairAgent, такие как сокращение системного запроса, агрессивное очищение контекста и загрузка инструментов по запросу, неожиданно увеличили стоимость работы агента из‑за потери кэширования и увеличения количества шагов.

Задача

Optimizing token usage in an AI agent can unintentionally increase cost by affecting cache hits and step count.

Результат

Each optimization increased cost by 67–82% because they broke cache continuity or added steps, demonstrating that token trimming alone can backfire.

Рабочий процесс

• Trim the system prompt to essential content
• Shorten tool descriptions to first sentence
• Clear old context aggressively (reduce budget to 40K)
• Introduce tool_output(id) to retrieve hidden outputs
• Add a note before clearing to preserve needed data
• Load tools by family instead of individually

Доказательства

The trimmed prompt made the agent 82% more expensive (6.49 m$ vs 3.56 m$).

Источник

Clearing the context budget from 120K to 40K tokens doubled the cost (58.6 m$ vs 30.6 m$).

Источник

Loading all 117 tools at once made a task 78% more expensive (10.95 m$ vs 6.15 m$).

Источник