Make the Failing Test the OSS Merge Token

Description and use

The workflow outlines creating a reproducible test that fails on main and passes on the fix branch, ensuring patches are validated before merging.

Problem

Open‑source patches stall because reviewers cannot reproduce the bug; patches often start as failing tests rather than diffs, leading to uncertainty about whether the issue still exists.

Outcome

The workflow ensures that every patch is accompanied by a reproducible test that fails on the current main and passes on the fix branch, reducing uncertainty and improving patch quality.

Workflow

• Freeze the observable claim in repro.json
• Add a hermetic failing test that reproduces the claim
• Run the reproduction command on main and capture output
• Create a branch for the issue and implement the minimal change
• Run the reproduction command on the fix branch and verify success
• Run the full test suite to ensure no regressions
• Automate fail‑on‑main and pass‑on‑branch checks with tools/check_repro.sh

Evidence

A recorded reproduction command closes that exact gap.

Source

The reproduction test must fail on current main. The same test must pass on the fix branch.

Source