Two failures stayed "unrepaired" after the fix had already shipped

Description and use

The automation pipeline’s ledger marked two failures as unrepaired even after the code fixes had merged, because the ledger only considered a failure repaired when a later run explicitly referenced the failed run in its repair_of array.

Problem

The automation pipeline’s ledger marked two failures as unrepaired even after the code fixes had merged, because the ledger only considered a failure repaired when a later run explicitly referenced the failed run in its repair_of array.

Outcome

The ledger correctly marked the failures as repaired only after a confirming run executed the affected paths, illustrating that code merges alone do not guarantee repair status without evidence of execution.

Workflow

• 1. Identify unrepaired failures in the ledger.
• 2. Review failure records to confirm root causes were fixed in merged PRs.
• 3. Observe that the ledger’s repair rule requires a later run to list the failed run’s ID in its repair_of array.
• 4. Trigger a run that exercises the affected paths.
• 5. Verify the run succeeds and records the failed run IDs in its repair_of array.
• 6. Confirm the ledger updates the failures to repaired.

Evidence

The ledger still listed two failures from the two days before as unrepaired, even though the pull requests that fixed their root cause had already merged.

Source

scripts/autopilot-selfheal.mjs defines "repaired" as exactly one thing: the failing run's ID appears in another run's repair_of array.

Source

Only at that point did both rows close. Nothing about the underlying bug changed that day — the code had already been correct since the two fixes merged, two days earlier.

Source