Should this prompt leave your laptop? Building a local-first AI router and doing the honest math

Description and use

GreenRoute is a local-first LLM router that decides whether to route a prompt to a local model or the cloud, estimating environmental savings. It routes 58 % of prompts locally, achieves 86 % acceptable quality, and cuts energy use by 48 %.

Problem

Users often send LLM requests to the cloud even when a small local model could answer them, wasting energy and privacy.

Outcome

GreenRoute kept 35 of 60 prompts locally, agreeing with manual labels 95% of the time. Local 1B model achieved 86% acceptable quality on routed prompts. Estimated environmental footprint reduced by 48% energy, 58% water, 19% CO₂e versus all-cloud baseline.

Workflow

• Classify request type and complexity
• Check device capabilities (model size, RAM, battery, power draw)
• Route to local model or cloud based on rules
• Show reasoning and estimated savings to user

Evidence

GreenRoute kept 35 of the 60 prompts on the device (58%) and agreed with my labels on 57 of 60.

Source

Compared with sending all 60 prompts to the cloud, GreenRoute's estimated footprint was 48% lower in energy (7.5 vs 14.4 Wh), 58% lower in water (6.5 vs 15.6 mL) and 19% lower in CO₂e (1.46 vs 1.80 g).

Source