A Noise Budget for Automated PR Reviews: Free Models, a Free Server, and a Protocol That Keeps Humans Listening

Описание и применение

Протокол «noise budget» ограничивает количество комментариев AI‑ревьюера до критических ошибок, сохраняя доверие и эффективность человеческих рецензентов при нулевых затратах.

Задача

Automated PR reviewers that comment on everything erode trust and overwhelm human reviewers, leading to rapid abandonment of the bot.

Результат

The protocol reduces noise to a manageable level, preserves bot credibility, and keeps human reviewers engaged while incurring near‑zero cost.

Рабочий процесс

• 1. Deploy a free model on a free server (e.g., MonkeyCode).
• 2. Configure a noise budget: set a maximum number of blocker comments per PR (NOISE_CAP).
• 3. Run the GitHub Action on pull_request events.
• 4. The Action fetches the base branch, calls the review endpoint, and pipes output to the severity‑gated runner.
• 5. The runner filters comments by severity, caps blocker comments, and outputs them in a checklist format.
• 6. Lower‑severity findings are added to the PR description as a quiet summary.
• 7. The bot’s comments are prefixed with its identity for auditability.
• 8. Track false positives in logs to adjust the noise budget over time.

Доказательства

The silent bot costs a token budget and nothing else; the chatty one trains a whole team to ignore every flag it raises, including the genuine blockers.

Источник

The runner consumes the model's raw review as JSON and reduces it to a short, accountable list.

Источник