I Built a CI Failure Digest on a Free Server. The Model Was the Easy Part.

Description and use

A daily bot fetches recent CI failures, classifies them as URGENT or WAIT using a free LLM, and only notifies on urgent issues, reducing Slack noise.

CI failures flood Slack, causing developers to ignore critical alerts.

The bot runs in under an hour, costs zero dollars, and effectively silences non‑urgent CI alerts while still notifying on critical failures.

1. Use GitHub CLI to fetch recent failed runs via gh run list --status failure --limit 20 --json ...
2. Filter out known noise workflows (docs, stale, flaky, timeout).
3. For each remaining run, construct a prompt for the LLM: "You triage CI failures… Reply with exactly one word: URGENT or WAIT."
4. Send the prompt to a free LLM endpoint (configured via environment variables LLM_BASE_URL, LLM_API_KEY, LLM_MODEL).
5. Collect the LLM response and classify the run as URGENT or WAIT.
6. If any runs are URGENT, print a concise list prefixed with "FIX NOW:"; otherwise print "Nothing urgent. Staying quiet."
7. Schedule the script to run daily via a cron job on a free server (e.g., a free tier VM or serverless function).

The bot pulls the last 24 hours of failed CI jobs, classifies each one, and posts a short summary.

If nothing is urgent, it stays quiet. That last part was the real requirement: silence is a feature, not a bug.