OpenClaw Horizontal Scaling Proposal

Описание и применение

Предложение по горизонтальному масштабированию OpenClaw включает внедрение Redis‑поддерживаемой очереди BullMQ для отделения обработки фоновых задач от основного приложения, что повышает масштабируемость, надёжность и наблюдаемость.

Задача

The current OpenClaw architecture tightly couples background job processing to the main application lifecycle, causing long-running tasks to block request handling, limited retry control, lack of delayed/cron job management, and difficulty scaling horizontally across multiple instances.

Результат

The prototype demonstrates that decoupling job processing via Redis-backed queues improves scalability, reliability, and observability, allowing OpenClaw to handle higher job volumes and run multiple instances without performance degradation.

Рабочий процесс

• 1. Install Redis and BullMQ dependencies in the OpenClaw project.
• 2. Refactor background task handlers to enqueue jobs into named BullMQ queues instead of executing inline.
• 3. Create worker scripts that subscribe to the queues and process jobs independently of the HTTP/API layer.
• 4. Configure job options for delay, repeatable (cron), priority, and retry/backoff strategies.
• 5. Implement a dead-letter queue for permanently failed jobs and ensure idempotent job handlers.
• 6. Add optional monitoring dashboard to visualize job states (waiting, active, completed, failed, delayed).
• 7. Deploy additional worker instances to scale horizontally without modifying application logic.

Доказательства

The prototype is available here: https://scalyclaw.netlify.app/

Источник

The main architectural change is the introduction of Redis and BullMQ to manage background jobs, delayed jobs, and cron-based tasks in a more structured way.

Источник