TrueNAS as an AI Ops Platform: Beyond File Storage to Inference, Images, and Pipelines

Description and use

Configuring TrueNAS SCALE as an AI Ops platform with GPU passthrough and deploying Ollama, ComfyUI, LiteLLM, and other services.

Problem

Deploying a full AI operations stack on a TrueNAS SCALE NAS while leveraging GPU passthrough and managing multiple services.

Outcome

The NAS now hosts a complete AI ops stack with GPU‑accelerated inference, image generation, vector search, workflow orchestration, and monitoring, all managed through TrueNAS SCALE apps and Docker containers.

Workflow

• Enable VM/Device passthrough in TrueNAS UI
• Identify GPU PCI IDs with lspci
• Configure Kubernetes or TrueNAS apps to passthrough GPU devices
• Install NVIDIA container toolkit if needed
• Deploy Ollama via TrueNAS app or docker run with GPU
• Organize model storage in /mnt/Storage_Pool/models and create symlinks
• Deploy ComfyUI via TrueNAS app or docker run with GPU
• Organize ComfyUI models in dedicated directories
• Deploy LiteLLM, Automatic1111, Open-WebUI, Qdrant, LangFuse, N8n, PostgresQL, Redis via TrueNAS apps or containers
• Set up NGINX Proxy Manager for SSL and routing
• Configure Fail2ban + CrowdSec for security
• Set up Netdata and Uptime Kuma for monitoring
• Create ZFS snapshots for rollback and experimentation

Evidence

TrueNAS SCALE includes built‑in GPU support for apps

Source

docker run --gpus all --rm nvidia/cuda:12.2-base nvidia-smi

Source