Why Dropping "await" is Broken: Building a Resilient Background Task Engine with Node.js, Redis Streams, and PostgreSQL

说明与用途

删除 await 会导致客户端连接被阻塞并产生重复操作。Mailman‑Relay 通过 Redis Streams 和 PostgreSQL 实现异步任务处理，具备原子去重、容错和零任务丢失功能。

问题

Holding the client’s HTTP connection hostage while executing slow external I/O causes UI freezes, duplicate actions, and unreliable side‑effects.

结果

Mailman‑Relay achieved ~592 req/s with 0.00% error, P50 ≈ 30 ms, P95 ≈ 50 ms, P99 ≈ 58 ms under 50 concurrent users, and zero job loss after worker crashes or network latency injection.

工作流

• Accept task via HTTP POST and validate input
• Acquire atomic idempotency lock with Redis SET NX EX
• Insert task metadata into PostgreSQL
• Append task payload to Redis Streams
• Return 202 Accepted to client
• Worker pool consumes stream entries, handles rate limits, retries, and acknowledges via XACK
• Supervisor thread uses XAUTOCLAIM to recover orphaned tasks

证据

When developers notice this latency, the most common "quick fix" is simply omitting the await keyword in Node.js

来源

Atomic Deduplication (Double-Click Shield) ... SET idempotency:{key} {job_id} NX EX {ttl}

来源

High-Concurrency Saturation (Grafana k6) ... Throughput: ~592 requests/second sustained

来源

Worker Crash Recovery (SIGKILL) ... Total jobs lost: Zero

来源