My Health Check Watched the Wrong File

说明与用途

基于日志文件年龄的健康检查规则因标准输出缓冲和写入不频繁而错误地将正在运行的代理标记为已死。通过检查状态文件和脚本行为发现了问题，并改进了规则。

问题

A health‑check rule that relied on log file age incorrectly flagged live Telegram‑polling agents as dead because the logs were block‑buffered and only written on errors or startup.

结果

The author’s initial health‑check rule incorrectly flagged live agents as dead because it relied on log file age, which was affected by stdout buffering and infrequent writes. By inspecting state files and understanding the scripts’ logging behavior, the author identified the flaw and suggested refining the rule to use more reliable artifacts.

工作流

• 1. Inspect log file mtimes for each agent.
• 2. Apply rule "Artifact age beats PID" to flag jobs with old logs as dead.
• 3. Observe that all three logs had mtime 2026‑08‑11 and mark jobs as dead.
• 4. Realize that the scripts write to stdout only on startup or errors, and stdout is block‑buffered.
• 5. Check state files; notice that state file mtimes are recent, indicating the jobs are alive.
• 6. Analyze script code to understand when logs are written (startup, errors, after handling a message).
• 7. Conclude that log file age is an unreliable signal for these agents.
• 8. Refine health‑check to use state file mtime or other reliable indicators, and consider adding -u to launch agents to avoid buffering.

证据

The log had not been touched since August 11 and the state file was seconds old.

来源

An unchanged file does not prove no errors occurred. It proves the file has not been modified.

来源

Both things are true at once: those programs have almost nothing to say, and without -u, the little they do say is not guaranteed to reach disk promptly.

来源