How I Cut My LLM API Costs by 70% Without Touching My Code

Description and use

Cut LLM API costs by 70 % by trimming the system prompt, summarizing chat history, and caching duplicate requests.

Problem

High monthly LLM API bill ($214.37) for a side project with no revenue, caused by inefficient request construction.

Outcome

Reduced token usage by trimming system prompt (~35% input token reduction), summarizing context (~50% reduction), and caching (~12% call reduction), achieving a 70% overall cost cut.

Workflow

• Count tokens in system and user prompts with tiktoken; trim system prompt from 1,200 to 180 tokens.
• Implement context summarization: keep last 4 messages, summarize older ones with a cheap model, reducing context tokens by ~60%.
• Add Redis cache: hash prompts, store responses with 1‑hour TTL; cache exact duplicates, cutting API calls by ~12%.
• Re‑run requests with new construction; observe bill drop to $61.80.

Evidence

I trimmed it down to 180 tokens. The core instructions—tone, format, constraints—stayed intact.

Source

This cut my context token usage by about 60% while keeping conversational continuity.

Source

I found that about 12% of my requests were exact duplicates. That's 12% of my bill I could just... not pay.

Source