One Line of mv Kills the 4x Duplicate Run After Recovery: Making launchd Queue Processing Idempotent

Описание и применение

Одно атомарное перемещение файла (mv) устраняет гонки в многопроцессорной среде launchd, гарантируя, что каждый JSON‑файл обрабатывается только одним процессом.

Задача

Multiple launchd lanes processed the same JSON files concurrently after a service outage, causing duplicate work and data corruption.

Результат

The atomic mv approach eliminated duplicate processing, ensuring idempotent queue handling and preventing data corruption after outages.

Рабочий процесс

• 1. Define DONE_DIR and PROCESSING_DIR variables. 2. Create PROCESSING_DIR if it doesn't exist. 3. For each JSON file in DONE_DIR: a. Attempt mv "$file" "$PROCESSING_DIR/$(basename "$file")". b. If mv succeeds, process the file. c. If mv fails (file already moved), skip to next file. 4. Continue with existing processing logic on the moved file.

Доказательства

Exactly one succeeds and the other is guaranteed to fail.

launchd simply invokes run-codex-funnel.sh according to this schedule.