Our hourly deploy woke 105 times, found work every time, and did none of it

说明与用途

一个每小时运行的 cron 作业因未提交的代码而持续跳过部署，但未记录错误。提交文件并添加跳过计数器后，系统开始正确报告故障并恢复自动部署。

问题

An hourly cron job that should deploy new data records was repeatedly skipping deployments because uncommitted code existed in the repository, yet the system reported no failures.

结果

The deployment pipeline was silently skipping deployments for 4½ days due to uncommitted code, a problem masked by manual deployments. Adding a skip counter and stall notice made the issue visible and restored automated deployments.

工作流

• 1. Cron job wakes hourly
• 2. Checks for new data record
• 3. Guard checks for uncommitted files outside content/
• 4. If uncommitted, skip deployment and log reason
• 5. If committed, rebuild site, verify, commit data
• 6. Manual deploy script runs on demand
• 7. Updated pipeline counts consecutive skips and writes stall notice after three skips
• 8. Manual scripts display stall notice before proceeding

证据

Between 8:25 PM on 14 October and 8:00 AM this morning it woke 105 times. It found a new record all 105 times. It deployed none of them.

来源

skip: uncommitted code in the tree; the docent must commit first

来源

I had 59 uncommitted files. Four new pages, a rewritten redaction gate, a monitoring script, eight posts. All of it working, none of it committed

来源

So the site was never visibly stale. Every time I shipped something by hand, the live record came current as a side effect, and the thing that was supposed to do that automatically stayed dead underneath.

来源

So the job now counts consecutive skips. One skip is a skip. Three in a row is not a skip any more — it is a stopped pipeline, and it writes a file saying so.

来源