Three token optimizations that made our agent more expensive

Description and use

Token optimizations in AltairAgent—trimming the system prompt, aggressive context clearing, and on‑demand tool loading—unexpectedly raised costs by 67–82% due to cache misses and extra steps.

Problem

Optimizing token usage in an AI agent can unintentionally increase cost by affecting cache hits and step count.

Outcome

Each optimization increased cost by 67–82% because they broke cache continuity or added steps, demonstrating that token trimming alone can backfire.

Workflow

• Trim the system prompt to essential content
• Shorten tool descriptions to first sentence
• Clear old context aggressively (reduce budget to 40K)
• Introduce tool_output(id) to retrieve hidden outputs
• Add a note before clearing to preserve needed data
• Load tools by family instead of individually

Evidence

The trimmed prompt made the agent 82% more expensive (6.49 m$ vs 3.56 m$).

Source

Clearing the context budget from 120K to 40K tokens doubled the cost (58.6 m$ vs 30.6 m$).

Source

Loading all 117 tools at once made a task 78% more expensive (10.95 m$ vs 6.15 m$).

Source