GPU Preflight For Cron Agents

Описание и применение

Функция preflight проверяет доступность GPU и его готовность к работе перед запуском cron‑задач, снижая вероятность неудачных запусков из‑за конкуренции ресурсов.

Задача

GPU-dependent cron jobs in shared environments often fail silently due to resource contention, leading to wasted compute hours.

Результат

The preflight check reduces silent GPU job failures, improves reliability, and minimizes wasted compute time, at the cost of a small additional latency from two HTTP requests.

Рабочий процесс

• 1. Send GET request to /api/tags with 3s timeout to confirm GPU accessibility.
• 2. Send POST request to /api/generate with a trivial prompt and 10s timeout to confirm GPU can handle a warmup request.
• 3. If both checks pass, proceed with the cron job; otherwise, skip execution and log failure.

Доказательства

We send a lightweight request to /api/tags, which should return within 3 seconds.

Источник

We send a trivial inference request to /api/generate, which should return within a short timeout window (e.g., 10 seconds).

Источник

This function returns False if either of the checks fails. In such cases, the cron job cleanly skips execution and logs the failure.

Источник