Real Cost Breakdown for LLM API Relays

Description and use

Cost analysis of an LLM relay revealed 92% savings driven by 87.5% prompt caching and a 0.08× price multiplier.

Problem

Determining the actual cost savings of using LLM API relays versus official vendor pricing.

Outcome

The relay achieved 92% cost reduction, largely due to 87.5% prompt cache hits and 0.08x group rate multiplier, resulting in an effective price of 7.97% of list.

Workflow

• 1. Gather billing dashboard data and request logs.
• 2. Extract metrics: requests, tokens, paid amount, list price, effective ratio.
• 3. Inspect usage block for prompt_cache_hit_tokens and prompt_cache_miss_tokens.
• 4. Identify group rate multipliers from console.
• 5. Compute cost savings from cache and multipliers.
• 6. Verify endpoint consistency and latency.
• 7. Perform sanity‑check steps: 401 test, /v1/models match, cache hit verification, cross‑check model output.

Evidence

I spent 28.67 CNY on 4.07 billion tokens: 92% reduction against list price.

Source

87.5% of the input was served from cache.

Source

The usage block in the response looks like this: { "usage": { "prompt_tokens": 1755, "prompt_cache_hit_tokens": 1536, "prompt_cache_miss_tokens": 219, "completion_tokens": 20 } }

Source

Many relays expose tiered rate multipliers per model group. In my case the open-weight group bills at 0.08x of list price, and a separate premium group at 0.22x.

Source