The Delivery ID Gap: Deduping Webhook Replays When the Provider Sends No ID

Описание и применение

Рецепт автоматизации в n8n, позволяющий обрабатывать вебхуки без идентификатора доставки, используя хеш‑fallback, отслеживая его частоту и адаптируя окна дедупликации и лимиты по отправителю.

Задача

Webhook providers sometimes omit a delivery ID, making it hard to deduplicate replayed events and detect accidental double-processing.

Результат

The recipe introduces a robust fallback mechanism for providers lacking delivery IDs, improves observability by counting fallback triggers, and adapts grace windows and rate limits based on evidence strength, reducing accidental double-processing while maintaining security.

Рабочий процесс

• 1. Verify the webhook signature to confirm sender authenticity.
• 2. Extract the delivery ID from headers or body; if absent, compute a deterministic hash of the timestamp-normalized payload plus sender key.
• 3. Generate a dedup key: use "sender:deliveryId" for strong keys, "sender:fb:hash" for weak keys.
• 4. Increment a per-provider fallback counter whenever the hash fallback is used.
• 5. Check the dedup store for the key; if present, throw a replay error.
• 6. Store the key with a TTL: full grace window for strong keys, shorter window for weak keys.
• 7. Apply a keyed rate limit based on the sender to mitigate hammering.
• 8. Monitor fallback rates per provider; when a provider stops triggering the fallback, switch to using the real delivery ID and tighten its defenses.
• 9. Review providers that never send IDs and route them to a review queue for manual inspection.

Доказательства

some of my providers don't send a delivery ID at all.

Источник

Be honest about what this buys: it kills accidental double-processing, not malicious replay.

Источник

increment a per-provider counter every time the hash fallback fires.

Источник

Shorter grace TTL. The dedup store TTLs at grace-window length plus clock skew; for fallback-only senders I shrink the window itself.

Источник