My 3-layer agent session locking fix stopped it from quietly wrecking customer records

Description and use

Implementing a three‑layer defense—per‑entity lock, idempotency keys, and ordered persistence—prevented silent data corruption from retries and parallel tool calls in an agent workflow.

Problem

Agent workflows that mutate external systems (HubSpot, Zendesk, etc.) can corrupt data when retries or parallel tool calls occur, even if a per-entity lock is used.

Outcome

The 3‑layer defense eliminated silent data corruption, ensuring that retries do not create duplicate external changes and that parallel tool calls do not overwrite each other.

Workflow

• Acquire per‑entity advisory lock
• Generate idempotency key at workflow boundary
• Store idempotency key before external request
• Reserve mutation in PostgreSQL table
• Execute external mutation if not already attempted
• Persist state with version check
• Release lock

Evidence

I learned this the ugly way at 2:13 a.m. when an n8n workflow reopened a closed Zendesk ticket, reassigned the account owner in HubSpot, and then did both again when the webhook retried.

Source

If your agent can write to HubSpot, Zendesk, Salesforce, Stripe, or your own internal APIs, a lock alone is not enough.

Source

I split tools into two classes: read-only and mutating. Then I allowed concurrency only for read-only calls.

Source