Real Cost Breakdown for LLM API Relays

Описание и применение

Разделённый анализ расходов на LLM‑релей показал 92% экономию благодаря 87,5% кэшированию запросов и 0,08× мультипликатору цены.

Задача

Determining the actual cost savings of using LLM API relays versus official vendor pricing.

Результат

The relay achieved 92% cost reduction, largely due to 87.5% prompt cache hits and 0.08x group rate multiplier, resulting in an effective price of 7.97% of list.

Рабочий процесс

• 1. Gather billing dashboard data and request logs.
• 2. Extract metrics: requests, tokens, paid amount, list price, effective ratio.
• 3. Inspect usage block for prompt_cache_hit_tokens and prompt_cache_miss_tokens.
• 4. Identify group rate multipliers from console.
• 5. Compute cost savings from cache and multipliers.
• 6. Verify endpoint consistency and latency.
• 7. Perform sanity‑check steps: 401 test, /v1/models match, cache hit verification, cross‑check model output.

Доказательства

I spent 28.67 CNY on 4.07 billion tokens: 92% reduction against list price.

Источник

87.5% of the input was served from cache.

Источник

The usage block in the response looks like this: { "usage": { "prompt_tokens": 1755, "prompt_cache_hit_tokens": 1536, "prompt_cache_miss_tokens": 219, "completion_tokens": 20 } }

Источник

Many relays expose tiered rate multipliers per model group. In my case the open-weight group bills at 0.08x of list price, and a separate premium group at 0.22x.

Источник