We put an ONNX background-removal container behind a Next.js Worker on Cloudflare

Description and use

The web app uses Cloudflare Workers, R2, and a Python container with an ONNX model for background removal. The architecture splits requests: POST enqueues a job, the client polls GET which runs inference within the CPU limit. The model loads in the background, weights are baked into the image, and file cleanup runs on a schedule.

Problem

Background removal in production requires handling queueing, cold starts, and CPU limits while keeping latency low.

Outcome

The architecture reduces cold‑start latency, respects Cloudflare’s 30s CPU limit, and provides reliable background‑removal service with scheduled cleanup.

Workflow

• 1. Client POST /api/tool/background-to-white uploads image, validates, stores in R2, creates job in D1, returns jobId.
• 2. Client polls GET /api/tool/background-to-white?jobId=… every 3s.
• 3. First poll that claims the job runs the ONNX container inference within a 20s long‑poll request.
• 4. Container loads model in background thread to satisfy health checks.
• 5. A warm endpoint (/api/warm) is called on page load to pre‑start the container during user idle time.
• 6. Cron jobs clean expired R2 uploads every 15min and resume orphaned jobs every minute.

Evidence

The container is a python:3.12-slim image with the model baked in at build time

Source

We set our CPU budget to 30,000 ms in wrangler.jsonc

Source

The port is ready in milliseconds, health checks pass, and the first real request just waits a little longer.

Source