MergeMeter: the ghosts of closed PRs predict which open ones will merge

Description and use

MergeMeter predicts merge probability for open or draft PRs by feeding the last 300 closed PRs of a public GitHub repo into TabPFN, achieving higher ROC‑AUC than logistic regression.

Problem

Contributors often submit pull requests that are closed without merging and never learn why, making it hard to gauge the likelihood of future PRs being merged.

Outcome

MergeMeter achieves a mean ROC‑AUC of 0.853 versus 0.824 for logistic regression across 8 real repos, indicating better predictive performance for merge likelihood.

Workflow

• 1. Input a public repo owner/repo string into MergeMeter.
• 2. MergeMeter fetches the repo’s last 300 closed PRs via the GitHub API.
• 3. The data is passed to TabPFN, an open‑weight tabular foundation model, which scores each open PR (or draft) with a merge probability and identifies key influencing features.
• 4. Results are displayed with visual tags (🌼, 🕯️, 👻) and a permutation importance chart.
• 5. Users can preview a draft PR’s probability before opening it.
• 6. Optionally run the app locally with python app.py or use the Hugging Face Docker Space.

Evidence

MergeMeter answers one anxious Hacktoberfest question: will this PR get merged?

Source

It pulls that repo's last 300 closed pull requests (the ones that lived, and the ghosts that were closed without merging) and hands them to TabPFN, an open-weight tabular foundation model.

Source

TabPFN reads that history in context (no training loop) and scores every open PR, or a draft you have not opened yet with a merge probability and the factors that decide a PR's fate in that repo.

Source

On 8 real repos (5‑fold CV), TabPFN beats logistic regression on 7 of 8: mean ROC‑AUC 0.853 vs 0.824.

Source

The app runs locally with python app.py; the repo is also a ready‑to‑push Hugging Face Docker Space if you have PRO.

Source