Spotify AI Routing Architecture for Claude Code

Описание и применение

Spotify снизил потребление токенов Claude Code на 90 % путём маршрутизации чтения больших файлов к более дешевому моделям и локальному генератору шаблонов.

Задача

High token consumption and costs when using Claude Code to read large code files.

Результат

Spotify reduced Claude Code token consumption by ~90% by routing large file reads to a cheaper model and handling boilerplate generation locally.

Рабочий процесс

• 1. Claude Code attempts to read a file.
• 2. Shunt intercepts the request and checks file size.
• 3. If file >350 lines, Shunt blocks the read and forwards the task to Bulk‑Reader Mode.
• 4. Bulk‑Reader (Gemini 2.5 Flash) reads the file, summarizes relevant parts, and returns a concise summary to Claude.
• 5. Claude performs deep reasoning on the summary.
• 6. For boilerplate code, Code‑Writer Mode generates and writes directly to disk without token‑heavy round‑trips.

Доказательства

routing simple tasks away from high‑end AI models to smaller, cheaper models decreased token consumption on the primary model by approximately 90%

Источник

Lack of Reasoning for Hard Bugs: Smaller worker models can identify simple patterns but miss nuanced flaws, such as concurrency bugs

Источник

Network Delay: Delegating tasks to a secondary service introduces a latency penalty of 10 to 30 seconds per request

Источник