I Sent 500 Requests to Auto-Routing. It Saved Me 47% and I Never Picked a Model

Описание и применение

Авто‑маршрутизация запросов к ИИ сократила расходы на 47 % и ускорила отклик, сохранив качество простых задач.

Задача

Using a single expensive model for all API requests led to unnecessary costs and latency.

Результат

Auto‑routing cut total cost by 47.1% and improved latency by 33%, while maintaining quality on simple tasks.

Рабочий процесс

• Collect 500 real prompts and categorize them into Simple, Medium, Complex.
• Send all prompts to DeepSeek‑V4 Pro (manual).
• Send all prompts with model="auto" (auto‑routing).
• Compare total cost, output tokens, and latency.
• Update code to set model="auto" and use force_complex=True only for complex tasks.
• Monitor usage dashboard after a week.

Доказательства

Total cost Manual $2.80, Auto‑routing $1.48 (-47.1%)

Источник

Auto‑router sent 300 simple to qwen3.8-flash, 150 medium to qwen3.8-max, 50 complex to deepseek-v4-pro

Источник