Ollama GPU Scheduling: Running Inference and ComfyUI on One RTX Without OOM

Description and use

Sharing a single RTX‑5060 Ti between Ollama and ComfyUI without OOM is achieved via time‑based or priority‑based GPU scheduling.

Problem

Share a single RTX 5060 Ti GPU between Ollama LLM inference and ComfyUI Stable Diffusion without exceeding 16 GB VRAM and avoiding out‑of‑memory errors.

Outcome

By allocating GPU time or priority, the 16 GB RTX 5060 Ti can run both Ollama and ComfyUI without OOM, enabling simultaneous code assistance and image generation.

Workflow

• 1. Measure VRAM usage of each model and pipeline to determine limits.
• 2. Choose a scheduling strategy: time‑based or priority‑based.
• 3. For time‑based, set systemd timers or cron jobs to grant/revoke GPU access at specific times.
• 4. For priority‑based, implement a GPU arbiter that queues requests with priority levels and preempts lower‑priority jobs.
• 5. Wrap Ollama and ComfyUI calls to request and release GPU access via the arbiter.
• 6. Monitor VRAM usage and adjust schedules or priorities as needed.

Evidence

Ollama VRAM Consumption (Approximate) ... Qwen2.5-Coder:14b (Q4_K_M): ~7.5 GB VRAM

Source

ComfyUI VRAM Consumption (Approximate) ... SDXL Base (1024x1024): ~6-8 GB VRAM

Source

Strategy 1: Time-Based Scheduling (Shift Work) ... Example: Daily schedule for balanced usage ... 0 6 * * * /usr/local/bin/gpu-grant ollama ...

Source

Strategy 2: Priority-Based Access (Reservation System) ... Pseudocode for GPU arbiter class ... requestAccess(application, priority, estimatedDuration)

Source