One Line of mv Kills the 4x Duplicate Run After Recovery: Making launchd Queue Processing Idempotent

Description and use

A single atomic mv operation removes race conditions in a launchd‑scheduled queue, ensuring each JSON file is processed only once.

Problem

Multiple launchd lanes processed the same JSON files concurrently after a service outage, causing duplicate work and data corruption.

Outcome

The atomic mv approach eliminated duplicate processing, ensuring idempotent queue handling and preventing data corruption after outages.

Workflow

• 1. Define DONE_DIR and PROCESSING_DIR variables. 2. Create PROCESSING_DIR if it doesn't exist. 3. For each JSON file in DONE_DIR: a. Attempt mv "$file" "$PROCESSING_DIR/$(basename "$file")". b. If mv succeeds, process the file. c. If mv fails (file already moved), skip to next file. 4. Continue with existing processing logic on the moved file.

Evidence

Exactly one succeeds and the other is guaranteed to fail.

launchd simply invokes run-codex-funnel.sh according to this schedule.