10 Tasks in Parallel: Killing the Queue Wait with orchestrate-codex-worker.sh

说明与用途

将10个Codex任务并行执行，将总运行时间从15分钟缩短到约90秒，提供实时状态并简化错误检测。

问题

Sequential execution of Codex tasks causes long idle CPU time and opaque failure visibility, delaying batch processing and making debugging hard.

结果

Parallel worker design reduces total batch time from ~15 minutes to ~90 seconds, eliminates idle CPU, provides real‑time status via status files, and simplifies failure detection and aggregation.

工作流

• Launch launchd job at scheduled times (08:00, 10:35).
• Run claude‑quota‑guard.py to ensure API quota.
• Invoke run‑and‑notify.sh to send start notification.
• Parent script article‑daily‑stock.sh starts N orchestrate‑codex‑worker.sh processes in background, each with task, handoff, and status files.
• Each worker writes "running" to its status file, builds prompt, calls codex exec with flags (-p yolo, -m gpt‑5.4, --color never, -C "$(pwd)", -o output_file).
• On success, worker writes structured handoff file (Summary, Files Changed, Validation, Remaining Risks) and updates status to "completed". On failure, writes failure summary and status "failed".
• Parent script waits for all workers, aggregates handoff files, generates report, and sends completion notification.

证据

Run 10 tasks in parallel and total elapsed time converges to "the slowest worker's processing time" + "startup overhead."

来源

Each worker keeps writing its own state to a status-file, the parent can read every job's progress even before it joins with wait.

来源