OpenHiggsfield Scaled Next.js Server Actions: Request Coalescing, 38 AI Models, and Zero Queue Locks

Описание и применение

В OpenHiggsfield реализован клиентский механизм коалессации запросов, который объединяет все активные задачи генерации в один пакетный запрос к серверу, устраняя сериализацию Next.js Server Actions и повышая производительность.

Задача

Next.js Server Actions serialize requests per client, causing queueing and UI stalls when multiple long‑running AI inference tasks poll for status.

Результат

Client‑side request coalescing reduced server‑side queueing, improved throughput, and kept the UI responsive even with 38 AI models and multiple concurrent generations.

Рабочий процесс

• 1. Register each generation request with watchRequest(requestId) in the client state machine.
• 2. The engine maintains a Map of waiting requests and a single timer tick every POLL_INTERVAL_MS.
• 3. On each tick, a batched Server Action getGenerationStatuses([id1,id2,…]) is dispatched.
• 4. Server Action handler performs concurrent fetches to AI providers using Promise.all.
• 5. When a terminal status is returned, the corresponding promise is resolved and removed from inflight.
• 6. If a request exceeds POLL_DEADLINE_MS or misses MAX_MISSES, it is rejected.
• 7. UI components await the promise returned by watchRequest to update status.
• 8. Repeat until all requests complete or fail.

Доказательства

Next.js serializes Server Action dispatches per client connection.

Источник

Client pools all pending generation IDs into a central state machine and dispatches a single batched Server Action per interval.

Источник