I stopped losing background jobs to sleep mode: a patient queue with leases

说明与用途

slothqueue 系统使用租约和延迟重试，确保在笔记本电脑进入睡眠模式或进程崩溃时后台任务不会丢失或重复。

问题

Background jobs on a laptop are lost or duplicated when the machine sleeps or the process crashes, because the job state does not distinguish between a slow job and a dead one.

结果

The slothqueue implementation prevents job loss or duplication on laptops by using leases and scheduled retries, enabling background jobs to survive sleep mode and crashes without extra infrastructure.

工作流

• 1. Worker calls claim() to reserve a job; the job’s lease_until is set to now + lease_duration and attempts is incremented.
• 2. Worker performs the job, periodically calling heartbeat() to extend the lease while the work is genuinely in progress.
• 3. Upon successful completion, worker calls complete() to mark the job as done and record a message.
• 4. If the worker crashes or the machine sleeps before completion, the lease expires; the next claim() will re‑issue the job with attempts == 2 and a reclaimed event.
• 5. If a job fails, the worker schedules the next attempt by setting not_before = now + 2**attempts (capped), so claim() only sees jobs whose not_before has passed.
• 6. When attempts reach max_attempts, the job is marked failed and no longer retried.
• 7. The system runs with a single SQLite database file, no network or broker, and no distributed workers.
• 8. Behaviour is verified by nine unit tests that inject a clock to test lease expiry, backoff, and failure handling without real sleeping.

证据

If the process dies between claim and complete, nothing needs to notice. The lease simply expires, and the next claim() re-issues the job with attempts == 2 and a reclaimed event in its history.

来源

failure schedules 2, 4, 8 seconds, capped, then lands in failed;

来源

python -m unittest discover -s tests -v # 9 tests, ~0.3s

来源

no distributed workers (a laptop doesn't need them — run two queues in two folders if you want two lanes); no parallelism; no dependencies at all — sqlite3 and subprocess from the standard library.

来源