Building a Leakage-Aware Readmission Model on 100,000 Diabetic Encounters

Description and use

A leakage‑aware readmission model achieved ROC‑AUC 0.691 and PR‑AUC 0.651, with a calibrated threshold yielding 69.7% sensitivity and 54.7% specificity, flagging 56.8% of patients for follow‑up.

Problem

Hospital readmission is costly and harmful; identifying patients likely to return before discharge can target scarce follow‑up resources.

Outcome

A leakage‑aware readmission model achieved ROC‑AUC 0.691 and PR‑AUC 0.651, with a calibrated threshold yielding 69.7% sensitivity and 54.7% specificity, flagging 56.8% of patients for follow‑up.

Workflow

• Load data with correct na_values and keep_default_na=False to preserve clinical categories
• Map integer missing codes using dataset ID mapping
• Define binary outcomes (any readmission, 30‑day readmission) and prediction time (end of stay)
• Exclude death/hospice encounters and post‑discharge variables via leakage review
• Create patient‑grouped StratifiedGroupKFold split ensuring no patient overlap
• Build scikit‑learn Pipeline: median imputation, standard scaling, one‑hot encoding with rare‑category pooling
• Fit logistic regression (baseline) and histogram gradient‑boosting models
• Evaluate metrics: ROC‑AUC, PR‑AUC, accuracy, calibration curve
• Select decision threshold on out‑of‑fold predictions to achieve ≥70% sensitivity
• Report sensitivity, specificity, precision, PR‑AUC, calibration, and threshold table
• Share code, figures, and report on GitHub

Evidence

If encounters from the same person land in both the training and test sets, the model can partly recognise the person instead of learning generalisable patterns.

Source

I used StratifiedGroupKFold to assign whole patients to an 80/20 train/test split while keeping the readmission rate balanced (0.471 in both), and asserted zero patient overlap.

Source

Logistic regression achieved a test ROC‑AUC of 0.679 (95% patient‑level bootstrap CI 0.670–0.687) and a PR‑AUC of 0.651 against a prevalence of 0.471.

Source

I chose the decision threshold on out‑of‑fold training predictions, not on the test set, using a stated rule: the highest threshold reaching at least 70% sensitivity. That gave 0.428.

Source