Ollama GPU Scheduling: Running Inference and ComfyUI on One RTX Without OOM

Описание и применение

Разделение одного RTX‑5060 Ti между Ollama и ComfyUI без ошибок OOM достигается через тайм‑шедулинг или приоритетный доступ к GPU.

Задача

Share a single RTX 5060 Ti GPU between Ollama LLM inference and ComfyUI Stable Diffusion without exceeding 16 GB VRAM and avoiding out‑of‑memory errors.

Результат

By allocating GPU time or priority, the 16 GB RTX 5060 Ti can run both Ollama and ComfyUI without OOM, enabling simultaneous code assistance and image generation.

Рабочий процесс

• 1. Measure VRAM usage of each model and pipeline to determine limits.
• 2. Choose a scheduling strategy: time‑based or priority‑based.
• 3. For time‑based, set systemd timers or cron jobs to grant/revoke GPU access at specific times.
• 4. For priority‑based, implement a GPU arbiter that queues requests with priority levels and preempts lower‑priority jobs.
• 5. Wrap Ollama and ComfyUI calls to request and release GPU access via the arbiter.
• 6. Monitor VRAM usage and adjust schedules or priorities as needed.

Доказательства

Ollama VRAM Consumption (Approximate) ... Qwen2.5-Coder:14b (Q4_K_M): ~7.5 GB VRAM

Источник

ComfyUI VRAM Consumption (Approximate) ... SDXL Base (1024x1024): ~6-8 GB VRAM

Источник

Strategy 1: Time-Based Scheduling (Shift Work) ... Example: Daily schedule for balanced usage ... 0 6 * * * /usr/local/bin/gpu-grant ollama ...

Источник

Strategy 2: Priority-Based Access (Reservation System) ... Pseudocode for GPU arbiter class ... requestAccess(application, priority, estimatedDuration)

Источник