A duplicate check inherits every delay of whoever answers it

Description and use

An unattended agent checked for duplicates by querying a remote listing API, but stale snapshots caused incorrect detections. The fix was to record the publish action locally within the same transaction, ensuring accurate and timely duplicate checks.

Problem

Unattended agents performed duplicate checks by querying a destination’s listing endpoint, but the endpoint returned stale snapshots, causing incorrect duplicate detection.

Outcome

Duplicate checks became accurate and timely, eliminating false positives caused by remote API staleness.

Workflow

• 1. Job produces output and writes a local record of the action in the same transaction.
• 2. Agent checks the local record to determine if the content has already been published.
• 3. If no local record exists, the agent proceeds to publish and then records the action.
• 4. The remote listing API remains available for other purposes but is no longer used for duplicate checks.

Evidence

We were not reading the truth. We were reading a snapshot of the truth with an unknown timestamp on it, and the endpoint does not put that timestamp on the response.

Source

What actually worked was moving the question to a different witness. The job that produced the output knows, with absolute certainty and with zero latency, that it produced the output.

Source