How I Cut My LLM API Costs by 70% Without Touching My Code

Описание и применение

Сокращение расходов на LLM‑API на 70 % за счёт сокращения системного запроса, суммирования истории чата и кэширования одинаковых запросов.

Задача

High monthly LLM API bill ($214.37) for a side project with no revenue, caused by inefficient request construction.

Результат

Reduced token usage by trimming system prompt (~35% input token reduction), summarizing context (~50% reduction), and caching (~12% call reduction), achieving a 70% overall cost cut.

Рабочий процесс

• Count tokens in system and user prompts with tiktoken; trim system prompt from 1,200 to 180 tokens.
• Implement context summarization: keep last 4 messages, summarize older ones with a cheap model, reducing context tokens by ~60%.
• Add Redis cache: hash prompts, store responses with 1‑hour TTL; cache exact duplicates, cutting API calls by ~12%.
• Re‑run requests with new construction; observe bill drop to $61.80.

Доказательства

I trimmed it down to 180 tokens. The core instructions—tone, format, constraints—stayed intact.

Источник

This cut my context token usage by about 60% while keeping conversational continuity.

Источник

I found that about 12% of my requests were exact duplicates. That's 12% of my bill I could just... not pay.

Источник