I Built an AI Orchestration System in My Basement. Here’s What Actually Worked.

Description and use

Five pipelines run 24/7 on a single mid‑range GPU with clear visibility, fast recovery, and minimal custom code.

Problem

Existing AI orchestration frameworks (LangChain, AutoGen, custom Python, Flowise, CrewAI, n8n) failed to provide clear visibility and reliability for production pipelines on a mid‑range GPU.

Outcome

Five pipelines run 24/7 on a single mid‑range GPU with clear visibility, fast recovery, and minimal custom code.

Workflow

• 1. Set up n8n for scheduling and data flow.
• 2. Configure Ollama Docker containers with a rotating model pool (llama3.1:8b, qwen2.5-coder:14b, deepseek-r1:14b).
• 3. Use LiteLLM as a unified proxy to switch models via a single config line.
• 4. Build n8n workflows that call Ollama REST endpoints for inference.
• 5. Store results in Postgres or SQLite; write memory embeddings to Qdrant.
• 6. Aggregate events into an activity feed via WebSocket to Postgres.
• 7. Deploy and monitor with ntfy alerts for NAS health and pipeline failures.