A timeout is not a failed webhook: reconcile before you retry

Description and use

When webhook timeouts occur, it is unclear if the action succeeded. The recipe reserves the operation with a key and hash, tracks state, and reconciles with the provider to avoid duplicates.

Problem

When a webhook request times out, it is unclear whether the provider applied the change, risking duplicate effects if retried.

Outcome

The recipe demonstrates local state transitions that prevent duplicate webhook actions after timeouts, using a reservation table and reconciliation via provider readback.

Workflow

• Generate a stable operation key and payload hash for each business operation.
• Persist the key, hash, and a 'dispatching' state in a durable store before calling the remote service.
• If a worker receives the same key, check existing state: if payload differs, treat as conflict; if same, skip sending again.
• After a timeout, mark the operation as 'unknown' and do not enqueue it for automatic retry.
• Reconcile by reading the provider’s state (using provider ID or stable identifier) to verify whether the effect already exists.
• If the effect matches, record the provider reference and mark the operation as 'verified'; otherwise keep it in 'unknown' for manual investigation.
• Only retry operations that are confirmed failures, not those with unknown outcomes.

Evidence

A webhook request times out. Should the worker retry?

Source

Give each business operation a stable key. Persist that key, the payload hash, and a dispatching state before calling the remote service.

Source

After a timeout that may have occurred after dispatch, retain the operation and mark it unknown. Do not put it back into the normal retry queue.

Source

Look up the original effect using a returned provider ID or a stable identifier that the provider actually supports.

Source