My Nightly Job Ran at 3 PM. Then the Disk Forgot, and the Model Returned Prose.

Описание и применение

Небольшой скрипт, запланированный через cron, запускался в неверное время, терял данные из-за эфемерного диска и получал некорректный вывод от бесплатного LLM. Добавлены проверки смещения часов, удалённые чекпоинты и обработка ошибок JSON.

Задача

A scheduled summarization job on a free server ran at the wrong time, lost its checkpoint data, and received malformed output from a free model endpoint.

Результат

The job now runs at the intended time, survives server restarts by persisting checkpoints remotely, and reliably parses JSON from the model, avoiding failures caused by malformed output.

Рабочий процесс

• 1. Add a clock drift check that compares local time to an HTTP Date header and exits if drift > 60s.
• 2. Integrate the clock check into the main script before any work is done.
• 3. Replace local checkpoint file with a remote checkpoint by sending a PUT request after each item.
• 4. Store the checkpoint URL in a secure location (e.g., object storage or a tiny endpoint).
• 5. Wrap model responses in a try/except block to catch JSON parsing errors.
• 6. If parsing fails, log the raw response and skip the item or retry.
• 7. Continue processing remaining items and update the remote checkpoint after each successful summary.

Доказательства

The free server had been suspended overnight, and when it resumed, its clock was still living in the afternoon.

Источник

The disk was erased on restart; the writable disk is ephemerally like /tmp.

Источник

The model returned prose instead of JSON, breaking the JSON parsing line.

Источник