A script that shows what your CI badge is actually testing

Description and use

The ci_truth.py script detects mismatches between CI workflow claims and actual execution conditions, enabling updates and more reliable badges.

Problem

CI badge claims often misrepresent actual test conditions, leading to unnoticed failures or outdated configurations.

Outcome

The script surfaces hidden CI misconfigurations, prompting timely updates and more accurate badges.

Workflow

• 1. Clone or run the script locally: python3 ci_truth.py owner/repo [branch]
• 2. The script fetches .github/workflows/*.yml via the GitHub API (no token needed for public repos).
• 3. It scans for Node version matrices, action usage, and install commands.
• 4. For each mismatch (EOL Node, outdated action, missing --frozen-lockfile), it outputs a human‑readable warning.
• 5. Review the output, update the workflow files, and commit changes to trigger a new CI run.

Evidence

The badge had been green for years.

Source

npm install instead of npm ci - lockfile drift cannot fail the build

Source