Real Cost Breakdown for LLM API Relays

说明与用途

对LLM中继的成本分析显示，通过87.5%的提示缓存和0.08×的价格乘数实现了92%的节省。

问题

Determining the actual cost savings of using LLM API relays versus official vendor pricing.

结果

The relay achieved 92% cost reduction, largely due to 87.5% prompt cache hits and 0.08x group rate multiplier, resulting in an effective price of 7.97% of list.

工作流

• 1. Gather billing dashboard data and request logs.
• 2. Extract metrics: requests, tokens, paid amount, list price, effective ratio.
• 3. Inspect usage block for prompt_cache_hit_tokens and prompt_cache_miss_tokens.
• 4. Identify group rate multipliers from console.
• 5. Compute cost savings from cache and multipliers.
• 6. Verify endpoint consistency and latency.
• 7. Perform sanity‑check steps: 401 test, /v1/models match, cache hit verification, cross‑check model output.

证据

I spent 28.67 CNY on 4.07 billion tokens: 92% reduction against list price.

来源

87.5% of the input was served from cache.

来源

The usage block in the response looks like this: { "usage": { "prompt_tokens": 1755, "prompt_cache_hit_tokens": 1536, "prompt_cache_miss_tokens": 219, "completion_tokens": 20 } }

来源

Many relays expose tiered rate multipliers per model group. In my case the open-weight group bills at 0.08x of list price, and a separate premium group at 0.22x.

来源