How I built an AI code reviewer that knows when to shut up

Описание и применение

Создано компактное решение для автоматизированного ревью кода, которое выводит только важные замечания, сортирует их по приоритету и сообщает о скрытых комментариях.

Задача

AI code reviewers often produce noisy, irrelevant feedback, causing developers to disable them and miss real bugs.

Результат

The system reduces noise, surfaces real bugs first, and maintains transparency by reporting suppressed comments, improving developer trust and adoption.

Рабочий процесс

• 1. Receive GitHub webhook for pull_request
• 2. Verify signature and immediately return 200
• 3. In background: fetch diff, call Claude to generate JSON findings
• 4. Validate severity against whitelist and sanitize findings
• 5. Stable‑sort findings by severity (BUG>WARN>NIT>PRAISE)
• 6. Slice to MAX_COMMENTS, compute suppressedCount
• 7. Post inline comments; if anchoring fails, fallback to top‑level comment
• 8. Log errors and suppress failures silently

Доказательства

GitHub wants a 2XX response within roughly 10 seconds of a webhook delivery.

Источник

The model returns JSON findings, each with { path, line, severity, message }

Источник