TrueNAS as an AI Ops Platform: Beyond File Storage to Inference, Images, and Pipelines

Описание и применение

Настройка TrueNAS SCALE как платформы AI Ops с GPU‑пропуском и развертыванием Ollama, ComfyUI, LiteLLM и других сервисов.

Задача

Deploying a full AI operations stack on a TrueNAS SCALE NAS while leveraging GPU passthrough and managing multiple services.

Результат

The NAS now hosts a complete AI ops stack with GPU‑accelerated inference, image generation, vector search, workflow orchestration, and monitoring, all managed through TrueNAS SCALE apps and Docker containers.

Рабочий процесс

• Enable VM/Device passthrough in TrueNAS UI
• Identify GPU PCI IDs with lspci
• Configure Kubernetes or TrueNAS apps to passthrough GPU devices
• Install NVIDIA container toolkit if needed
• Deploy Ollama via TrueNAS app or docker run with GPU
• Organize model storage in /mnt/Storage_Pool/models and create symlinks
• Deploy ComfyUI via TrueNAS app or docker run with GPU
• Organize ComfyUI models in dedicated directories
• Deploy LiteLLM, Automatic1111, Open-WebUI, Qdrant, LangFuse, N8n, PostgresQL, Redis via TrueNAS apps or containers
• Set up NGINX Proxy Manager for SSL and routing
• Configure Fail2ban + CrowdSec for security
• Set up Netdata and Uptime Kuma for monitoring
• Create ZFS snapshots for rollback and experimentation

Доказательства

TrueNAS SCALE includes built‑in GPU support for apps

Источник

docker run --gpus all --rm nvidia/cuda:12.2-base nvidia-smi

Источник