My Health Check Watched the Wrong File

Описание и применение

Правило проверки живости процессов, основанное на времени изменения лог‑файла, неверно определило живые Telegram‑пуллеры как мертвые из‑за блок‑буферизации вывода. Проверка файлов состояния показала активность.

Задача

A health‑check rule that relied on log file modification time incorrectly flagged live Telegram‑poller jobs as dead because the logs were block‑buffered and only written on errors or startup.

Результат

The health‑check rule was too simplistic; log file age alone is insufficient when stdout is block‑buffered and logs are only written on errors or startup. State file mtimes and contents are more reliable indicators of activity for these polling scripts.

Рабочий процесс

• 1. Apply the rule to the machine and observe false positives.
• 2. Inspect the log files and note they are unchanged.
• 3. Examine the scripts to see when they write to stdout and to state files.
• 4. Realise that stdout is block‑buffered unless -u is used.
• 5. Check state file mtimes and contents to confirm activity.
• 6. Conclude that the rule must be refined to consider the specific artifact and its write semantics.

Доказательства

A job that is loaded, looks healthy to the supervisor, and is producing nothing is the failure worth catching.

Источник

The log had not been touched since August 11 and the state file was seconds old.

Источник

An unchanged file does not prove no errors occurred. It proves the file has not been modified.

Источник

Those three logs will still carry mtime 2026-08-11 on 2026-10-01, while the processes are still alive.

Источник