I made CodeRabbit's reviews a third less noisy with an open-source Claude Code skill

Описание и применение

pr-proof — открытая реализация навыка Claude Code, которая проверяет каждый комментарий кода, уменьшая шум в обзорах CodeRabbit на 34 % и повышая точность F1 до 40,4 %.

Задача

AI code review bots generate many noisy comments that developers skim, causing real bugs to slip through.

Результат

pr-proof achieved a 93.5% retention of real bugs, removed 34% of noise issues, and raised CodeRabbit’s F1 score by 5.2 points (95% CI +1.9 to +8.3).

Рабочий процесс

• Install pr-proof via Claude Code plugin marketplace
• Run pr-proof on a PR to validate each review comment
• pr-proof reads the full file, traces execution, checks against conventions and docs
• It returns a verdict (valid, partly valid, wrong, style) citing file and line
• After all comments are verified, pr-proof applies approved fixes and replies on threads

Доказательства

On a public benchmark of 50 real pull requests, CodeRabbit raised 300 issues. 77 of them were real bugs on the benchmark's list. Most of the rest were noise: things that look like problems but aren't once you read the code.

Источник

kept 72 of the 77 real bugs (93.5%)

Источник

removed 76 of the 223 noise issues (34%)

Источник

raised CodeRabbit's F1 score from 35.2% to 40.4% (+5.2 points, 95% CI +1.9 to +8.3)

Источник