Ollama GPU Scheduling: Running Inference and ComfyUI on One RTX Without OOM

说明与用途

通过时间或优先级调度，在单个 RTX‑5060 Ti 上同时运行 Ollama 和 ComfyUI，避免 OOM。

问题

Share a single RTX 5060 Ti GPU between Ollama LLM inference and ComfyUI Stable Diffusion without exceeding 16 GB VRAM and avoiding out‑of‑memory errors.

结果

By allocating GPU time or priority, the 16 GB RTX 5060 Ti can run both Ollama and ComfyUI without OOM, enabling simultaneous code assistance and image generation.

工作流

• 1. Measure VRAM usage of each model and pipeline to determine limits.
• 2. Choose a scheduling strategy: time‑based or priority‑based.
• 3. For time‑based, set systemd timers or cron jobs to grant/revoke GPU access at specific times.
• 4. For priority‑based, implement a GPU arbiter that queues requests with priority levels and preempts lower‑priority jobs.
• 5. Wrap Ollama and ComfyUI calls to request and release GPU access via the arbiter.
• 6. Monitor VRAM usage and adjust schedules or priorities as needed.

证据

Ollama VRAM Consumption (Approximate) ... Qwen2.5-Coder:14b (Q4_K_M): ~7.5 GB VRAM

来源

ComfyUI VRAM Consumption (Approximate) ... SDXL Base (1024x1024): ~6-8 GB VRAM

来源

Strategy 1: Time-Based Scheduling (Shift Work) ... Example: Daily schedule for balanced usage ... 0 6 * * * /usr/local/bin/gpu-grant ollama ...

来源

Strategy 2: Priority-Based Access (Reservation System) ... Pseudocode for GPU arbiter class ... requestAccess(application, priority, estimatedDuration)

来源