Fortifying the Fortress: A Practical Guide to Network Isolation for Self-Hosted AI Models

Описание и применение

Практическое руководство по изоляции сети для самодостаточных ИИ‑моделей: привязка к localhost, TLS‑прокси nginx, аутентификация и правила ufw для защиты от внешних угроз.

Задача

Self‑hosted AI models often expose their API on 0.0.0.0, making them discoverable and vulnerable to attacks such as prompt injection, RCE, and cost‑exploitation.

Результат

AI service becomes isolated from the public network, encrypted in transit, authenticated, and protected by firewall rules, reducing exposure to attacks.

Рабочий процесс

• 1. Bind AI server to 127.0.0.1 (e.g., vLLM: --host 127.0.0.1).
• 2. Configure nginx as reverse proxy with TLS termination and proxy_pass to localhost:8000.
• 3. Add authentication in nginx (auth_basic or auth_request).
• 4. Set firewall rules to deny external traffic to port 8000 and allow only from loopback.
• 5. Test connectivity from external client via nginx endpoint.

Доказательства

Deploying a large language model or AI inference engine on your own infrastructure is a powerful move... The default configuration of many AI frameworks often binds the serving kernel directly to a public network interface or 0.0.0.0

Источник

Nginx acts as a hardened, public-facing frontend that terminates external TLS connections and securely forwards traffic to your localhost-bound AI kernel.

Источник

Using ufw on Ubuntu, this would be: # Deny all traffic to the AI port by default sudo ufw deny in on any to any port 8000 # Allow only from Nginx running on the same machine sudo ufw allow in on lo to any port 8000

Источник