Replacing LiteLLM in Docker (500 MB RAM) with an 11 MB Go binary: fast LLM fallbacks and OS keyring

说明与用途

OperatorLM 是一个轻量级 Go 代理，替代 500 MB Docker LiteLLM 容器，提供快速多提供商回退，使用系统 keyring 存储 API 密钥，简化本地 LLM 开发。

问题

Running LiteLLM in Docker consumes ~500 MB RAM and requires a Python runtime, while managing API keys via scattered .env files is insecure and repetitive.

结果

OperatorLM reduces resource usage from 500 MB to ~11 MB, eliminates scattered .env files, provides intelligent failover across providers, and logs all requests in a JSONL audit file.

工作流

• Start OperatorLM listening on 127.0.0.1:11434
• Configure aliases mapping a model name to ordered provider targets
• Store provider API keys in the OS keyring and reference them in config.toml
• Send OpenAI‑compatible requests to OperatorLM
• OperatorLM resolves the alias, injects the key, retries or fails over according to strategy, logs the outcome
• Optional: use the embedded admin UI for configuration and monitoring

证据

A single 11 MB Go binary can configure multi-provider fallbacks instantly, store your API keys securely in your OS keyring instead of scattered .env files, and dramatically simplify your developer life.

来源

LiteLLM does this, but running it in Docker took about 500 MB of RAM on my machine just to route local HTTP traffic.

来源

Keys go into the keyring your OS already has: Windows Credential Manager, macOS Keychain, or Linux Secret Service.

来源