I Built a Production-Ready Job Queue in Go from Scratch

Описание и применение

Создана продакшн‑готовая очередь задач на Go, использующая Postgres вместо Redis, с обработкой повторных попыток, ограничением скорости и корректным завершением работы.

Задача

Develop a background job queue that handles retries, rate limiting, and graceful shutdown without relying on external services like Redis.

Результат

A production-ready job queue that eliminates the need for Redis, demonstrates ACID compliance, and showcases Go concurrency best practices.

Рабочий процесс

• 1. Client POST /jobs to Gin API
• 2. API writes job to Postgres jobs table
• 3. Worker pool (5 goroutines) fetches pending jobs using FOR UPDATE SKIP LOCKED
• 4. Worker processes job and updates status
• 5. On failure, retry with exponential backoff or mark as failed after max attempts
• 6. Graceful shutdown via context cancellation

Доказательства

A REST API where you POST a task, it goes to Postgres, and Go workers process it in background with retries, rate limiting and graceful shutdown.

Источник

Postgres as queue (no Redis needed)

Источник

Worker Pool with 5 Goroutines

Источник

Retry with exponential backoff

Источник