Building Sarrera: Self-Hosted Enterprise AI Inference Gateway with RBAC, Token Quotas & Telemetry

说明与用途

Sarrera 让企业能够在本地运行生成式 AI，提供细粒度 RBAC、令牌配额和实时遥测，消除云端隐私风险和不可控费用，同时高效利用异构 GPU/CPU 资源。

问题

Engineering teams face privacy, billing, and hardware fragmentation challenges when adopting generative AI.

结果

Sarrera enables enterprises to run generative AI locally with fine‑grained RBAC, token quotas, and real‑time telemetry, eliminating cloud privacy risks and uncontrolled billing while efficiently utilizing heterogeneous GPU/CPU resources.

工作流

• Deploy Sarrera via Docker Compose
• Configure Caddy for TLS and reverse proxy
• Set up LiteLLM with model routing and quota tiers
• Deploy Langfuse, Open WebUI, MinIO, PostgreSQL, and Ollama nodes
• Use Admin Control Panel to register nodes and issue API keys
• Integrate with VS Code by adding Sarrera model to config.json

证据

Engineering teams worldwide face a common dilemma when adopting generative AI: Cloud AI privacy & security risks, uncontrolled cloud billing, hardware fragmentation

来源

Sarrera decouples client IDEs from physical compute hardware, shielding your internal network behind a single perimeter reverse proxy

来源

LiteLLM checks if the requested model is whitelisted for their tier. If a junior developer with tier-basic requests premium-reasoning, the gateway immediately returns HTTP 403 Forbidden in 10 milliseconds

来源