I wouldn’t trust an AI agent to cron alone anymore — 8 reliability rules I’d ship by Friday

Description and use

Using a queue and idempotency keys for unattended AI agents eliminates duplicate actions, prevents job overlap, and improves observability.

Problem

Unattended AI agents in production often fail due to duplicate triggers, overlapping jobs, missed cron runs, and silent crashes, leading to repeated side effects such as duplicate emails or charges.

Outcome

Implementing a queue‑based architecture with idempotency keys and worker‑level retries reduced duplicate side effects, prevented overlapping jobs, and improved observability of unattended AI agents.

Workflow

• 1. Cron or webhook enqueues a job into a durable queue (e.g., BullMQ, SQS).
• 2. The queue stores the job with a unique idempotency key and retry configuration.
• 3. Workers pull jobs from the queue and execute agent steps.
• 4. Each step performs LLM or API calls within retriable logic.
• 5. Before any side effect, the worker checks or creates an idempotency record to prevent duplicates.
• 6. Successful jobs are removed or archived; failed jobs are retried or sent to a dead‑letter queue.
• 7. Monitoring verifies that each trigger actually occurred and alerts on silent failures.

Evidence

Cron should create work, not do work.

Source

Duplicate suppression only works while the old job still exists.

Source

The pattern I like is simple: orchestration decides what should happen next; worker steps do the nondeterministic work.

Source