My Nightly Job Ran at 3 PM. Then the Disk Forgot, and the Model Returned Prose.

说明与用途

一个通过 cron 调度的摘要脚本在错误的时间运行，因磁盘是临时的而丢失了检查点，并且从免费 LLM 收到非 JSON 的输出。解决方案包括时钟漂移检查、远程检查点存储和健壮的 JSON 验证。

问题

A scheduled summarization job on a free server ran at the wrong time, lost its checkpoint data, and received malformed output from a free model endpoint.

结果

The job now runs at the intended time, survives server restarts by persisting checkpoints remotely, and reliably parses JSON from the model, avoiding failures caused by malformed output.

工作流

• 1. Add a clock drift check that compares local time to an HTTP Date header and exits if drift > 60s.
• 2. Integrate the clock check into the main script before any work is done.
• 3. Replace local checkpoint file with a remote checkpoint by sending a PUT request after each item.
• 4. Store the checkpoint URL in a secure location (e.g., object storage or a tiny endpoint).
• 5. Wrap model responses in a try/except block to catch JSON parsing errors.
• 6. If parsing fails, log the raw response and skip the item or retry.
• 7. Continue processing remaining items and update the remote checkpoint after each successful summary.

证据

The free server had been suspended overnight, and when it resumed, its clock was still living in the afternoon.

来源

The disk was erased on restart; the writable disk is ephemerally like /tmp.

来源

The model returned prose instead of JSON, breaking the JSON parsing line.

来源