What does a green CI check actually prove?

Description and use

Problem: a green CI badge does not guarantee that tests ran on the correct commit and covered all changed files. Solution: CI Evidence Gate binds evidence to the exact SHA, verifies producer, coverage, and policy, producing a clear verdict.

Problem

A green CI badge may not guarantee that the correct tests ran on the correct commit under a trustworthy policy, leading to merge decisions based on incomplete or detached evidence.

Outcome

By binding CI evidence to the exact commit, producer, coverage, and policy provenance, the CI Evidence Gate prevents stale or detached green results from enabling merges, improving security and reliability in complex CI setups.

Workflow

• 1. Clone the repository and run the synthetic pattern FFA‑001 to reproduce a stale green CI scenario.
• 2. Observe that the fixture reports a vulnerable gate accepting a detached receipt.
• 3. Identify missing bindings: subject identity, producer identity, coverage, policy provenance, and freshness.
• 4. Define a policy manifest mapping changed surfaces to required checks and enforce fail‑closed on unmapped files.
• 5. Implement the CI Evidence Gate as a read‑only GitHub Action that queries GitHub APIs for checks, verifies metadata, and outputs a receipt.
• 6. Use the receipt to make merge decisions based on evidence sufficiency rather than a single green badge.

Evidence

GitHub can associate a check run with a commit SHA, and branch protection can require a check from a selected GitHub App.

Source

The evidence must name the exact pull‑request head SHA being evaluated.

Source

The safest default is fail‑closed: an unmapped changed file is a finding, not a silent exemption.

Source