One Bad Message Kept Killing My Queue: Poison Messages and Dead‑Letter Queues in n8n

Описание и применение

Один некорректный запрос блокировал очередь в n8n. Внедрение классификации ошибок, ограниченных повторов, DLQ и механизма повторного выполнения восстановило нормальную работу очереди.

Задача

A single malformed message kept blocking a production queue in n8n, causing the backlog to grow each day while the queue appeared healthy.

Результат

The queue stopped growing, poison messages were isolated in a DLQ, and replays after fixes processed previously failed jobs, restoring normal queue behavior.

Рабочий процесс

• 1. Detect job failure in workflow execution.
• 2. Classify failure as transient (timeout, 429, 5xx) or permanent (schema violation, missing field, parse error).
• 3. If transient, retry with backoff up to a small cap; if cap reached, move to DLQ with reason "retries exhausted."
• 4. If permanent, move job directly to DLQ with full payload, error, attempt count, and timestamp.
• 5. DLQ stores entries with structured failure reason and alerts on new entries (deduplicated by failure kind).
• 6. When issue is fixed, replay DLQ entry: re‑queue as new job with same payload, link original failure, and record replay job ID.
• 7. If replay succeeds, mark DLQ entry resolved; if it fails again, return to DLQ with new attempt count and cap retries for replay attempts.

Доказательства

A workflow queue looked healthy from the outside: new jobs arriving, executions running, no errors on the dashboard. But the backlog never shrank.

Источник

The job itself was not special. It was one malformed message — a payload with a field type the workflow did not expect — and every time a worker picked it up, the workflow threw, the job went back to the queue, and the next worker picked it up again.

Источник

Permanent failures go to a dead-letter queue (DLQ): a separate store that holds the failed job, the full original payload, the error, the attempt count, and the time it failed.

Источник

Replay is a first-class operation, not a manual hack. A replay takes a DLQ entry, re-queues it as a new job with the same payload, and records the link between the original failure and the replay attempt.

Источник