Replayable Automation Needs Better Run Receipts

Description and use

Automation becomes more transparent and debuggable thanks to compact JSON receipts that capture identity, inputs, decisions, attempts, evidence, and cleanup, enabling quick identification and replay of failures.

Problem

Automation failures are hard to diagnose because run logs lack concise, actionable context, making it difficult to understand why a job failed or how to reproduce the issue.

Outcome

Automations become more transparent and debuggable; developers can quickly locate a run, understand every decision, and replay the scenario with controlled fixtures, reducing mystery and improving reliability.

Workflow

• 1. Execute automation job with unique run ID and commit SHA.
• 2. Capture safe inputs (feature flags, endpoint names).
• 3. Log decisions (continue, retry, stop) and attempt details (start/end time, status, correlation ID).
• 4. Record evidence (response status, headers, redacted IDs).
• 5. Note cleanup actions (data removed or left).
• 6. Store the receipt as a JSON artifact in CI logs.
• 7. Use the receipt to replay decisions with safe fixtures and compare outcomes.

Evidence

A receipt turns an automation run from a single outcome into something a developer can replay and understand.

Source

Start with one JSON file per run. Add richer tracing only when the receipt shows a real gap.

Source