Why Dropping "await" is Broken: Building a Resilient Background Task Engine with Node.js, Redis Streams, and PostgreSQL

Описание и применение

Удаление await приводит к блокировке клиентского соединения и дублированию действий. Mailman‑Relay, использующий Redis Streams и PostgreSQL, обеспечивает асинхронную обработку задач с атомарной дедупликацией, устойчивостью к сбоям и нулевыми потерями задач.

Задача

Holding the client’s HTTP connection hostage while executing slow external I/O causes UI freezes, duplicate actions, and unreliable side‑effects.

Результат

Mailman‑Relay achieved ~592 req/s with 0.00% error, P50 ≈ 30 ms, P95 ≈ 50 ms, P99 ≈ 58 ms under 50 concurrent users, and zero job loss after worker crashes or network latency injection.

Рабочий процесс

• Accept task via HTTP POST and validate input
• Acquire atomic idempotency lock with Redis SET NX EX
• Insert task metadata into PostgreSQL
• Append task payload to Redis Streams
• Return 202 Accepted to client
• Worker pool consumes stream entries, handles rate limits, retries, and acknowledges via XACK
• Supervisor thread uses XAUTOCLAIM to recover orphaned tasks

Доказательства

When developers notice this latency, the most common "quick fix" is simply omitting the await keyword in Node.js

Источник

Atomic Deduplication (Double-Click Shield) ... SET idempotency:{key} {job_id} NX EX {ttl}

Источник

High-Concurrency Saturation (Grafana k6) ... Throughput: ~592 requests/second sustained

Источник

Worker Crash Recovery (SIGKILL) ... Total jobs lost: Zero

Источник