How I Cut My LLM API Costs by 70% Without Touching My Code

说明与用途

通过缩短系统提示、摘要聊天历史和缓存重复请求，将LLM API成本降低70%。

问题

High monthly LLM API bill ($214.37) for a side project with no revenue, caused by inefficient request construction.

结果

Reduced token usage by trimming system prompt (~35% input token reduction), summarizing context (~50% reduction), and caching (~12% call reduction), achieving a 70% overall cost cut.

工作流

• Count tokens in system and user prompts with tiktoken; trim system prompt from 1,200 to 180 tokens.
• Implement context summarization: keep last 4 messages, summarize older ones with a cheap model, reducing context tokens by ~60%.
• Add Redis cache: hash prompts, store responses with 1‑hour TTL; cache exact duplicates, cutting API calls by ~12%.
• Re‑run requests with new construction; observe bill drop to $61.80.

证据

I trimmed it down to 180 tokens. The core instructions—tone, format, constraints—stayed intact.

来源

This cut my context token usage by about 60% while keeping conversational continuity.

来源

I found that about 12% of my requests were exact duplicates. That's 12% of my bill I could just... not pay.

来源