Exit Code 0, Nothing Produced: The Automation Failure I Shipped Five Times

Описание и применение

Автоматические задания, завершающиеся кодом 0, но не создающие выходных данных, обнаружены благодаря объявлению ожидаемого вывода в JSON‑файле и внешнему измерению состояния до и после выполнения.

Задача

Automated jobs repeatedly succeeded with exit code 0 yet produced no output, going unnoticed for weeks until a human review caught the issue.

Результат

By declaring expected outputs and measuring them externally, the team detected silent failures, reduced wasted wall time, and improved reliability of the data pipeline.

Рабочий процесс

• Define each job in a shared JSON file with a "produces" block specifying output type, path, max_age_h, and description.
• Wrap each job with an external runner that records pre‑run state via a measure() function.
• Execute the job and capture exit code, stdout, stderr.
• Measure post‑run state with measure() again.
• Compute the delta or freshness based on output type and compare against expectations.
• Log the result and update the ledger entry.
• Alert if the delta/freshness indicates a failure or if the job exits with a non‑zero status.
• Implement failure handling rules such as consecutive‑failure caps and per‑job wall‑clock ceilings.
• Adjust output_expectation to differentiate between "every_run" and "conditional" scenarios.
• Iterate and refine monitoring rules to avoid false positives.

Доказательства

I moved the definition of every scheduled job into one JSON file, and required each entry to carry a produces block — a machine-readable claim about what must exist after a healthy run.

Источник

The runner measures the declared output twice: once before the child process starts, once after it exits.

Источник