Your dedupe key is fine. Your dedupe state is the problem.

Description and use

The bot duplicated posts because dedupe state was stored locally. The fix was to use a composite key and enforce single‑machine queueing, with plans to move state to a shared store.

Problem

The bot announced duplicate articles because dedupe state was stored locally and not shared across machines, causing idempotency to fail.

Outcome

The bot now correctly handles per‑account deduplication and prevents duplicate posts across machines by enforcing a single‑machine queueing guard and planning to move state to a shared store.

Workflow

• 1. Collect today's article URLs.
• 2. Filter URLs against posted.json and queue.json using a composite key (webhook|url).
• 3. Append unique entries to queue.json.
• 4. Process queue: send entries whose scheduled time has arrived and move them to posted.json.
• 5. If running on multiple machines, enforce a single‑machine queueing guard via an environment variable; fail closed if unset.

Evidence

Both machines evaluated their own records correctly. Both reached a correct conclusion. The announcement went out twice, and posts cannot be unsent.

Source

The real fix is moving that state somewhere shared: Firestore, object storage, a shared filesystem.

Source