The Agent Patch Test Budget: Spend CI Minutes Where They Catch Faults

Description and use

AI‑generated patches are cheap to produce but costly to validate. By classifying patches by risk and limiting CI test time per risk level, the pipeline processes more patches while still catching faults.

Problem

Agent-generated patches are cheap to produce but expensive to validate; running the same full test gauntlet on every patch throttles CI throughput.

Outcome

By allocating CI minutes based on patch risk, the pipeline processes more patches per hour, reducing bottlenecks while still catching faults where they matter.

Workflow

• 1. Agent generates patch and pushes to CI
• 2. CI runs gate_budget.sh to classify patch risk and output gate plan
• 3. CI executes only the gates specified in the plan within the allocated budget
• 4. Flaky freeze runs outside the budget to quarantine unstable tests
• 5. If a gate times out, the patch fails and the agent is prompted to split the diff

Evidence

Classify each patch by risk, then assign a time budget and a gate set.

Source

A gate that times out fails the patch.

Source