Replacing LiteLLM in Docker (500 MB RAM) with an 11 MB Go binary: fast LLM fallbacks and OS keyring

Description and use

OperatorLM is a lightweight Go proxy that replaces the 500 MB Docker LiteLLM container, offers fast multi‑provider fallbacks, stores API keys in the OS keyring, and simplifies local LLM development.

Problem

Running LiteLLM in Docker consumes ~500 MB RAM and requires a Python runtime, while managing API keys via scattered .env files is insecure and repetitive.

Outcome

OperatorLM reduces resource usage from 500 MB to ~11 MB, eliminates scattered .env files, provides intelligent failover across providers, and logs all requests in a JSONL audit file.

Workflow

• Start OperatorLM listening on 127.0.0.1:11434
• Configure aliases mapping a model name to ordered provider targets
• Store provider API keys in the OS keyring and reference them in config.toml
• Send OpenAI‑compatible requests to OperatorLM
• OperatorLM resolves the alias, injects the key, retries or fails over according to strategy, logs the outcome
• Optional: use the embedded admin UI for configuration and monitoring

Evidence

A single 11 MB Go binary can configure multi-provider fallbacks instantly, store your API keys securely in your OS keyring instead of scattered .env files, and dramatically simplify your developer life.

Source

LiteLLM does this, but running it in Docker took about 500 MB of RAM on my machine just to route local HTTP traffic.

Source

Keys go into the keyring your OS already has: Windows Credential Manager, macOS Keychain, or Linux Secret Service.

Source