Exit Code 0, Nothing Produced: The Automation Failure I Shipped Five Times

Description and use

Automated jobs that succeeded with exit code 0 yet produced no output were detected by declaring expected outputs in a JSON schema and externally measuring state before and after execution.

Problem

Automated jobs repeatedly succeeded with exit code 0 yet produced no output, going unnoticed for weeks until a human review caught the issue.

Outcome

By declaring expected outputs and measuring them externally, the team detected silent failures, reduced wasted wall time, and improved reliability of the data pipeline.

Workflow

• Define each job in a shared JSON file with a "produces" block specifying output type, path, max_age_h, and description.
• Wrap each job with an external runner that records pre‑run state via a measure() function.
• Execute the job and capture exit code, stdout, stderr.
• Measure post‑run state with measure() again.
• Compute the delta or freshness based on output type and compare against expectations.
• Log the result and update the ledger entry.
• Alert if the delta/freshness indicates a failure or if the job exits with a non‑zero status.
• Implement failure handling rules such as consecutive‑failure caps and per‑job wall‑clock ceilings.
• Adjust output_expectation to differentiate between "every_run" and "conditional" scenarios.
• Iterate and refine monitoring rules to avoid false positives.

Evidence

I moved the definition of every scheduled job into one JSON file, and required each entry to carry a produces block — a machine-readable claim about what must exist after a healthy run.

Source

The runner measures the declared output twice: once before the child process starts, once after it exits.

Source