I spent a day auditing GitHub Actions across a dozen AI agent frameworks

Description and use

Audited CI workflows in a dozen AI agent frameworks, found misconfigurations, and opened PRs for fixes.

Problem

CI workflows in popular AI agent projects often contain misconfigurations, unpinned actions, and silent regressions that go unnoticed, compromising reliability and security.

Outcome

Half of the examined repositories had at least one workflow issue; PRs were opened for remediation, raising awareness of CI hygiene in the AI community.

Workflow

• 1. Enumerate target repositories
• 2. Clone each repo and inspect .github/workflows
• 3. Check for unpinned actions and mutable tags
• 4. Verify exit codes and failure handling
• 5. Identify silent reverts of security guards
• 6. Document findings and create PRs
• 7. Notify maintainers and track review status

Evidence

Microsoft's autogen had a CI check meant to catch out of sync generated proto files. It could never fail.

Source

griptape had an action pinned to a mutable tag instead of a commit hash, tj-actions/changed-files at v47.

Source