Mining PR Review Comments to Surface Team’s Unwritten Code‑Review Rules

Description and use

PR Rulebook scans PR review comments to surface undocumented review rules, ranking them by confidence for human validation.

Problem

Teams often have implicit code‑review conventions that are never documented, making it hard for new members and automated tools to enforce consistent standards.

Outcome

The pilot run on 15 merged PRs from ruff yielded 45 review comments and two candidate rules, revealing limitations in the initial clustering logic and leading to improved evidence filtering across multiple PRs.

Workflow

• Clone the PR Rulebook repo
• Install dependencies and build the CLI
• Set a read‑only GitHub token
• Run the CLI against a target repo for the last 6 months
• Review the generated REVIEW_RULES.md for candidate rules and evidence

Evidence

The v0 clustering emitted two candidate rules.

Source

The fix now requires evidence across at least two distinct PRs.

Source