OpenHiggsfield Scaled Next.js Server Actions: Request Coalescing, 38 AI Models, and Zero Queue Locks

Description and use

OpenHiggsfield introduced a client‑side request coalescing engine that batches all active generation tasks into a single Server Action per interval, eliminating Next.js serialization bottlenecks and improving throughput.

Problem

Next.js Server Actions serialize requests per client, causing queueing and UI stalls when multiple long‑running AI inference tasks poll for status.

Outcome

Client‑side request coalescing reduced server‑side queueing, improved throughput, and kept the UI responsive even with 38 AI models and multiple concurrent generations.

Workflow

• 1. Register each generation request with watchRequest(requestId) in the client state machine.
• 2. The engine maintains a Map of waiting requests and a single timer tick every POLL_INTERVAL_MS.
• 3. On each tick, a batched Server Action getGenerationStatuses([id1,id2,…]) is dispatched.
• 4. Server Action handler performs concurrent fetches to AI providers using Promise.all.
• 5. When a terminal status is returned, the corresponding promise is resolved and removed from inflight.
• 6. If a request exceeds POLL_DEADLINE_MS or misses MAX_MISSES, it is rejected.
• 7. UI components await the promise returned by watchRequest to update status.
• 8. Repeat until all requests complete or fail.

Evidence

Next.js serializes Server Action dispatches per client connection.

Source

Client pools all pending generation IDs into a central state machine and dispatches a single batched Server Action per interval.

Source