The Crash Test Nobody Runs: What I Learned About "Exactly-Once" Building AI Workflows in n8n

Description and use

By moving idempotency handling outside n8n and using a dedicated job table with atomic claims, the author achieved true exactly‑once semantics for providers that enforce uniqueness, and correctly flagged uncertain cases for human review, eliminating duplicate emails and other side effects.

Problem

n8n workflows can finish successfully while external side effects (emails, charges, database rows) are duplicated or lost, because n8n only reports execution completion, not external action idempotency.

Outcome

By moving idempotency handling outside n8n and using a dedicated job table with atomic claims, the author achieved true exactly‑once semantics for providers that enforce uniqueness, and correctly flagged uncertain cases for human review, eliminating duplicate emails and other side effects.

Workflow

• 1. Add a local HTTP relay between n8n and the provider to simulate failure modes (fail-before-forward, forward-then-drop-response, forward-and-respond).
• 2. Count side effects at the provider or relay log, not at n8n execution count.
• 3. Persist a self‑contained job row with a stable deduplication key derived from the source event.
• 4. Atomically claim the job using an INSERT … ON CONFLICT statement to avoid race conditions.
• 5. Perform the external side effect only if the claim succeeds.
• 6. Mark the job as done with an UPDATE that only succeeds if status is pending.
• 7. Run a scheduled sweeper to recover jobs stuck in claimed state beyond a timeout.
• 8. Classify jobs in the sweeper: for key‑enforced providers, retry or mark done; for key‑advisory providers, move to review and alert.
• 9. Run two tests: (a) concurrent identical deliveries to verify atomic claim; (b) inject failure between side effect and mark‑done to verify sweeper lands in review.

Evidence

n8n can tell you a workflow ran. It cannot tell you the side effect happened exactly once.

Source

The request reached the provider and the side effect committed, but your process never saw the response - connection reset after commit, timeout after the write landed, process killed between send and receive.

Source

Put a tiny local HTTP endpoint in front of the provider (or a mock that still records acceptance), and select the behavior with a header or env var: ...

Source

Persist a self-contained job row - a stable dedup key derived from the source event (never $execution.id, $now, or a fresh uuid)...

Source

INSERT INTO idem (idem_key, status, claimed_at) VALUES ($1, 'pending', now()) ON CONFLICT (idem_key) DO NOTHING RETURNING idem_key;

Source

After the sweeper runs, class 2 must land in Unknown/review, never auto-retry.

Source