The Webhook Burst That Buried My Queue: Backpressure in n8n

Description and use

Implementing a lightweight webhook handler, bounded worker, priority queue, and backpressure signals reduced latency and prevented stale deliveries during event bursts.

Problem

A webhook workflow received a sudden burst of over a thousand events in one minute, causing downstream 429 responses, queue backlogs, and stale deliveries.

Outcome

Implementing a fast, lightweight webhook handler with a bounded worker, priority queue, and backpressure signals reduced latency, prevented stale deliveries, and enabled graceful degradation during bursts.

Workflow

• 1. Webhook handler validates signature and deduplicates by delivery ID.
• 2. Enqueue payload to durable queue and return 200.
• 3. If queue depth exceeds derived threshold, return 429 with Retry-After.
• 4. Worker pulls a fixed number of jobs (e.g., 2) concurrently.
• 5. Worker processes jobs in priority order, calling downstream APIs.
• 6. Completed jobs are marked done; refused jobs are logged with delivery ID.
• 7. Alerting monitors projected staleness (depth/drain rate) and triggers before backlog burial.

Evidence

In sixty seconds it received over a thousand.

Source

The webhook handler does almost nothing. It validates the signature, checks the delivery id for duplicates, writes the payload to a durable queue, and returns 200.

Source

The worker pulls a fixed number of jobs at a time. Mine is small — two concurrent executions for a provider with a documented rate limit.

Source

When the queue depth passes a threshold, the webhook handler stops returning 200 and starts returning 429 with a Retry-After header.

Source

My queue rows carry a priority, and the worker claims the highest-priority oldest job first.

Source