The most expensive part of my agent stack wasn’t tokens — it was the invoice roulette

Description and use

In production agent stacks, cost is driven by workflow behavior—retries, fallbacks, background jobs, and caching—rather than token count alone. Recognizing this enables strategic use of batch APIs and prompt caching to control expenses.

Problem

Agent stacks that run 24/7 incur unpredictable and often higher costs than token‑based forecasts suggest, because workflow behaviors such as retries, fallbacks, background jobs, and caching significantly affect billing.

Outcome

Recognizing that billing volatility stems from workflow design rather than token count led to a shift in cost‑management strategy: use OpenAI Batch for offline jobs, apply Anthropic caching only when prompts are stable, and separate async work for Gemini batch. This reduces unexpected spikes and aligns cost with architectural choices.

Workflow

• 1. Receive event (e.g., support ticket webhook).
• 2. AI Agent classifies urgency via LLM call.
• 3. Agent calls CRM tool.
• 4. Agent calls knowledge base tool.
• 5. Agent drafts reply via LLM call.
• 6. If a tool call times out, retry the step.
• 7. On retry, earlier steps may run again, increasing token usage.
• 8. Background jobs (nightly summarization, cache refresh) run independently of user events.

Evidence

If you run agents 24/7, especially across n8n, Make, Zapier, OpenClaw, or custom workflows, this matters more than most pricing pages admit.

Source

OpenAI Batch is a solid option for offline work. 50% lower cost than synchronous API usage.

Source

Anthropic caching looks fantastic on paper. It can be a huge win if your prompt prefixes are stable and requests arrive inside the cache window.

Source