We put an ONNX background-removal container behind a Next.js Worker on Cloudflare

Описание и применение

Веб‑приложение использует Cloudflare Workers, R2 и Python‑контейнер с ONNX‑моделью для удаления фона. Архитектура разделяет запросы: POST создаёт задачу, клиент опрашивает GET, который выполняет инференс внутри лимита CPU. Модель загружается в фоне, веса встроены в образ, и очистка файлов выполняется по расписанию.

Задача

Background removal in production requires handling queueing, cold starts, and CPU limits while keeping latency low.

Результат

The architecture reduces cold‑start latency, respects Cloudflare’s 30s CPU limit, and provides reliable background‑removal service with scheduled cleanup.

Рабочий процесс

• 1. Client POST /api/tool/background-to-white uploads image, validates, stores in R2, creates job in D1, returns jobId.
• 2. Client polls GET /api/tool/background-to-white?jobId=… every 3s.
• 3. First poll that claims the job runs the ONNX container inference within a 20s long‑poll request.
• 4. Container loads model in background thread to satisfy health checks.
• 5. A warm endpoint (/api/warm) is called on page load to pre‑start the container during user idle time.
• 6. Cron jobs clean expired R2 uploads every 15min and resume orphaned jobs every minute.

Доказательства

The container is a python:3.12-slim image with the model baked in at build time

Источник

We set our CPU budget to 30,000 ms in wrangler.jsonc

Источник

The port is ready in milliseconds, health checks pass, and the first real request just waits a little longer.

Источник