Should this prompt leave your laptop? Building a local-first AI router and doing the honest math

Описание и применение

GreenRoute – локальный маршрутизатор LLM, который решает, отправлять ли запрос в облако или обрабатывать его на устройстве, и оценивает экономию энергии и CO₂. Он маршрутизирует 58 % запросов локально, достигая 86 % удовлетворительного качества и снижая энергопотребление на 48 %.

Задача

Users often send LLM requests to the cloud even when a small local model could answer them, wasting energy and privacy.

Результат

GreenRoute kept 35 of 60 prompts locally, agreeing with manual labels 95% of the time. Local 1B model achieved 86% acceptable quality on routed prompts. Estimated environmental footprint reduced by 48% energy, 58% water, 19% CO₂e versus all-cloud baseline.

Рабочий процесс

• Classify request type and complexity
• Check device capabilities (model size, RAM, battery, power draw)
• Route to local model or cloud based on rules
• Show reasoning and estimated savings to user

Доказательства

GreenRoute kept 35 of the 60 prompts on the device (58%) and agreed with my labels on 57 of 60.

Источник

Compared with sending all 60 prompts to the cloud, GreenRoute's estimated footprint was 48% lower in energy (7.5 vs 14.4 Wh), 58% lower in water (6.5 vs 15.6 mL) and 19% lower in CO₂e (1.46 vs 1.80 g).

Источник