Manaz Afzal
Healthcare · Post-discharge follow-up calls

The same nurse hours, spent on the patients most likely to come back

A nightly call list that ranks discharges by risk of readmission, tested on 14,811 stays of patients the model never saw.

35%of readmissions reached by calling 20% of discharges; LACE-style 27%
+28%more returning patients reached for the same calls
0.64AUC on later patients; LACE-style 0.58

Executive summary

Ranking discharges by readmission risk reaches 35% of 30-day readmissions with calls to one discharge in five, against 27% for a LACE-style score: 28% more returning patients for the same nurse hours.

35%of readmissions reached by calling 20% of discharges, against 27%
+28%more returning patients reached for the same calls
48more at-risk patients called a year at 600 discharges a month

Situation

Follow-up calls after discharge are limited by nurse hours, and the call list is built from a rule of thumb or a paper score that misses many of the patients who come back.

Approach

A gradient-boosted risk model on fields already in the discharge record, trained on 57,521 patients and tested on 12,469 later patients against a LACE-style score and prior stays.

Result

At the same call capacity the list reaches 34.7% of readmissions instead of 27.2%. AUC 0.64 against 0.58: a modest signal that pays off because it re-orders a fixed list.

The problem

After discharge, a hospital’s transition-care nurses phone patients to check medicines, symptoms and follow-up appointments. There are only so many calls a shift can make, so most services pick patients with a LACE-style score or by ward. Published programmes that pair risk screening with nurse calls show lower readmissions in the patients they reach, while calls made without targeting show little effect, which makes the choice of who to call the part that matters.

I built a nightly list that ranks discharges by readmission risk and measured how many 30-day readmissions it reaches at the same number of calls as a LACE-style score.

Team
Discharge or transitional-care nurses, ward managers, patient flow
Systems
Cerner or Epic discharge records, the patient administration system, a call worklist in the nursing system
Measure
Share of 30-day readmissions whose patient was on the call list

What I built

A nightly risk list. Each discharge from the day is scored from details already in the discharge record: where the patient went, prior inpatient and emergency visits, main diagnosis, length of stay, medicines and age. The list is sorted highest risk first and cut at the number of calls the team can make.

DischargeRecord closed inthe PASScoreRisk of returnwithin 30 daysCall listSorted, cut attoday’s capacityNurse callMedicines, symptoms,next appointment
The score only decides the order of the list. The nurse still decides what each call covers.

Results

On 14,811 stays of 12,469 patients the model never saw, calling the highest-risk 20% of discharges reaches 34.7% of everyone readmitted within 30 days. The LACE-style score reaches 27.2% with the same number of calls, so the same nurse hours reach 28% more of the patients who come back.

Calling at random20%Prior inpatient stays29%LACE-style score27%Risk model35%
0%25%50%75%100%10%20%30%40%50%60%70%80%90%100%Calling at randomLACE-style scorePrior inpatient staysRisk modelCalling 20%: 35% ofreadmissions reachedShare of discharges called, highest risk first
The model stays ahead of the LACE-style score at every call capacity. AUC is 0.64 (95% range 0.62 to 0.65) against 0.58 for the score and 0.55 for prior stays alone.
Share of discharges calledRisk modelLACE-style scorePrior staysReadmitted among those called (model)
10%22.1%15.8%16.3%19.5%
20%34.7%27.2%29.4%15.3%
30%46.4%38.5%39.5%13.6%

This is a modest signal, and I report it as one. Readmission depends on much that a discharge record does not hold: who is at home, whether the scripts were filled, how the patient feels on day three. The gain comes from ranking a fixed list better, not from predicting individuals with confidence.

Calibration

PredictedObservedLowest tenth4%Second tenth5%Third tenth5%Fourth tenth7%Fifth tenth8%Sixth tenth8%Seventh tenth10%Eighth tenth10%Ninth tenth11%Highest tenth20%
Predicted risk tracks observed risk across the tenths. The model runs slightly high because later patients in this data have fewer recorded prior visits (8.8% readmitted in the test period against 11.8% in training), so the score is recalibrated on a hospital’s own recent discharges before use.

What drives the ranking

Where the patient was discharged to0.063Inpatient stays in the prior year0.031Main diagnosis group0.010Admitting specialty0.007Length of stay0.004Emergency visits in the prior year0.004Diagnoses recorded0.004Age band0.002
Discharge destination and prior inpatient stays carry most of the signal; diagnosis and specialty refine it.
No inpatient stay in the prior year11,503 stays8%One2,345 stays11%Two640 stays15%Three or more323 stays18%
Patients with three or more inpatient stays in the prior year are readmitted at more than twice the rate of those with none.
20-30 years223 stays6%30-40 years522 stays8%40-50 years1,323 stays7%50-60 years2,554 stays8%60-70 years3,557 stays9%70-80 years3,589 stays10%80-90 years2,549 stays10%90-100 years410 stays12%
Risk rises with age, but less steeply than prior use of the hospital.
Main diagnosis groupTest stays30-day readmission
Circulatory4,3399.6%
Other2,6968.8%
Respiratory1,9757.6%
Digestive1,3757.9%
Diabetes1,1249.5%
Injury1,08310.2%
Genitourinary8678.2%
Musculoskeletal8216.5%
Neoplasms5299.6%

Data and method

Data
Diabetes 130-US Hospitals for years 1999–2008 (Strack et al., 2014; UCI Machine Learning Repository, CC BY 4.0): 101,766 inpatient stays of patients with diabetes. Stays ending in death or hospice discharge are excluded, leaving 99,343
Outcome
Readmitted within 30 days of discharge
Split
By patient. Patients first seen in the earliest 80% of stays (57,521 patients, 84,532 stays) train the model; patients first seen later (12,469, 14,811 stays) are the test set, so no patient is in both
Model
Gradient-boosted trees (scikit-learn HistGradientBoosting) on discharge destination, admission type and source, diagnosis group, specialty, length of stay, lab and medicine counts, prior visits, HbA1c and glucose results, insulin and age
Baselines
A LACE-style score built from the fields available (length of stay, emergency admission, a diagnosis-count proxy for comorbidity, emergency visits) and prior inpatient stays alone
Uncertainty
Bootstrap 95% range on AUC over test stays
Brier score
0.0787 against 0.0814 for a constant rate

Limitations

  • The data comes from United States hospitals between 1999 and 2008 and covers patients with diabetes only. An Australian health service would train and test on its own discharges before any list goes live.
  • The LACE-style baseline lacks a true Charlson comorbidity index, so it understates a full LACE score somewhat.
  • Readmissions to another hospital are not visible in this data, as in most single-service records.
  • A call list sorted by risk changes who gets called, so outcomes of called patients are tracked separately from the model’s accuracy.

How I would take it live

  • Pull two years of discharges and readmissions from the patient administration system
  • Retrain, recalibrate and agree the daily call capacity with the transitional-care team
  • Run the list in shadow beside the current rule for a month and compare who each would have called
  • Switch over, then report weekly on readmissions among called and uncalled patients

Technical depth

Patients are split by their first stay so that repeat stays of one patient never straddle training and test. Categorical inputs are passed to the trees as native categories. Importances are permutation drops in AUC on held-out stays. No patient-level details are shown on this page.

Analysis code (Python)

Financial impact

About 48 more at-risk patients called a year, with no extra nurse hours

The gain is in who gets the calls the team already makes. Any fall in readmissions depends on what the calls achieve, so that part is an assumption.

InputValueSource
Discharges a month600 mid-sized hospital or service
Share of discharges called20% today’s nurse capacity
30-day readmission rate8.8%Measured test period
Readmissions reached, risk model34.7%Measured
Readmissions reached, LACE-style score27.2%Measured
Readmissions avoided per extra patient called15% effect of a follow-up call
Cost of a readmissionA$6,000 about one national efficient price unit

Calculation

  1. Readmissions a month53
  2. Reached with the LACE-style list14.4
  3. Reached with the risk list18.4
  4. Extra at-risk patients called a year48
  5. Readmissions avoided a year7.1

Annual value of readmissions avoidedA$42,876

The value rests on the assumed effect of a call. The share of returning patients reached is measured; the rest is measured only after go-live.

Implementation and rollout

Ten weeks from discharge export to a live call list

  1. Weeks 1 to 3

    Export and baseline

    Two years of discharges and readmissions from the patient administration system; measure who today’s rule calls and who comes back.

    Health information manager, Manaz

  2. Weeks 4 to 5

    Train and calibrate

    Train on the earlier patients, test on the latest, recalibrate and agree daily capacity with the nurse unit manager.

    Manaz, nurse unit manager

  3. Weeks 6 to 9

    Shadow list

    The risk list runs beside the current rule; the team calls as usual and both lists are compared each week.

    Transitional-care nurses

  4. Week 10 onward

    Live list

    The nightly list sets the call order. Weekly report of readmissions among called and uncalled patients; quarterly retraining.

    Nurse unit manager, Manaz on support

Tools

Nightly extract from the patient administration system or EMR, Python (scikit-learn) scoring job inside the health service’s own environment, Call list delivered to the existing scheduling or worklist tool, Weekly readmission report by call status

Team

  • Nurse unit manager owns the call capacity and signs off go-live
  • Transitional-care nurses make the calls and flag patients the list misses
  • Health information manager data access and privacy approvals
  • Manaz data work, model, integration and monitoring

The timeline is a plan for a typical engagement; it depends on data access and how many systems are involved.

Risks and mitigations

What could go wrong in production, and the control for each

RiskMitigation
Patient data leaving the serviceScoring runs inside the health service’s environment on de-identified extracts; nothing is sent to outside services.
A clinician’s concern overrides the listNurses can add any patient to the day’s calls; additions are logged and used in retraining.
Bias against groups with less recorded historyReach by age, sex and diagnosis group is reported each month and the cut is reviewed if any group falls behind.
Drift after changes to discharge practiceCalibration is checked monthly against observed readmissions.

Key takeaways

Key takeaways

  • For the same number of calls, the risk list reaches 28% more of the patients who come back.
  • The signal is modest; its value comes from ordering a list the team already works through.
  • Discharge destination and prior inpatient stays carry most of the signal.
  • The model is retrained and recalibrated on the service’s own discharges before it sets a single call.