Education · Colleges and training providers
Students at risk of leaving, flagged while there is still time to help
A termly list for student-support advisers: who to contact first, and why. Tested on 4,424 undergraduate records. After the first semester, the top fifth of the list contains 57% of the students who later dropped out, and 92% of the students on it did leave.
Executive summary
After the first semester, the top fifth of a termly list holds 57% of the students who later dropped out, and 92% of the students on it did leave.
Situation
Two or three advisers support a few thousand students and hear about trouble when fees go overdue or a census date passes, which is often too late.
Approach
Two scores from one pipeline, at enrolment and after semester one, on 4,424 undergraduate records, scored out-of-fold; gender, nationality and marital status excluded.
Result
The semester-one score reaches 57% of dropouts in a fifth of the intake against 39% for the fee rule. The enrolment score is weaker and arrives a semester earlier.
The business problem
Student-support advisers at a university or TAFE can contact only part of each intake. Early-alert programmes such as Georgia State’s, which prompt an adviser when a student’s record shows a risk, are credited with higher retention and graduation, but they depend on alerts that are few enough to act on and arrive before the student has gone.
I measured how many eventual dropouts a termly adviser list catches at enrolment and again after first-semester results, at a list size an advising team can work, against the fees-overdue rule many institutions use.
- Team
- 2 to 3 student-support advisers
- Scale
- An intake of 500 to 5,000 students a year
- Systems
- Student record system such as Callista, TechnologyOne Student or Banner, a CRM for advising such as Salesforce Education Cloud
- Constraint
- Privacy Act 1988 and state education rules; flags inform an adviser and never change grades, fees or enrolment
What was built
Two scores from the same pipeline. The first runs at enrolment from application details, prior qualifications, admission grade, attendance mode, age and fee status. The second runs after first-semester results, adding units enrolled, assessed and passed, and grades. Each produces a ranked list with plain reasons ("passed 2 of 6 units; fees overdue; evening attendance") in the adviser's queue in the student management system or a shared sheet.
Results
| List | Students flagged | Dropouts caught | Flagged who left |
|---|---|---|---|
| Enrolment score, top 10% | 442 | 29.2% | 93.9% |
| Enrolment score, top 20% | 884 | 48.5% | 77.9% |
| Semester-one score, top 10% | 442 | 30.7% | 98.6% |
| Semester-one score, top 20% | 884 | 57.5% | 92.4% |
| Fees overdue or debtor | 785 | 39.0% | 70.6% |
| Random 20% | 885 | 20.0% | 32.1% |
AUC is 0.822 at enrolment and 0.903 after semester one, against 0.656 for the fee rule. Including the excluded personal attributes would raise the semester-one AUC only to 0.906.
The signal in first-semester results
What it means for a provider
For an intake of 1,000 with this dropout rate, about 321 students will leave. The semester-one list of 200 contains about 185 of them. If adviser contact keeps one in five of those enrolled for another year, that is about 37 students; at an annual fee of $15,000 that is roughly $550,000 in tuition.
The retention effect of the contact itself is not measured in this data. A provider should test it: contact half of the flagged list in the first term and compare re-enrolment with the other half.
Data and method
- Source
- UCI Machine Learning Repository 697, "Predict Students' Dropout and Academic Success" (Realinho et al., 2022)
- Size
- 4,424 students: 2,209 graduated, 794 still enrolled, 1,421 dropped out
- Setting
- Polytechnic Institute of Portalegre, Portugal; 17 undergraduate degrees; intakes 2008/09 to 2018/19
- Validation
- 5-fold stratified cross-validation, 3 seeds, out-of-fold scores
Gradient-boosted classifiers predict dropout against graduating or still being enrolled. The enrolment score uses 20 attributes; the semester-one score adds six first-semester attributes. Second-semester data is held back entirely so that the second score is available mid-year.
Limitations
- One Portuguese institution. Course mix, fee structures and student support differ in Australia, so the model is refitted on the provider's own three to five years of records
- Cross-validation mixes intakes; a stricter test trains on earlier intakes and tests on later ones, which the provider's own data allows
- Macroeconomic fields (unemployment, inflation, GDP) are national values at enrolment and act as an intake-year marker
- A high-precision list can still miss quiet leavers who pass their units and then do not re-enrol
What a provider would do next
- Weeks 1 to 2: extract three to five years of enrolments, results and withdrawals from the student management system; agree the privacy impact assessment
- Weeks 3 to 4: refit on earlier intakes, test on the latest; check the list with advisers
- Next term: run the list with a split contact trial
- End of year: measure re-enrolment against the trial split and decide
Technical depth
The two checkpoints share preprocessing so their scores are comparable. Excluded attributes were tested both ways and reported, so the cost of leaving them out is visible. Precision at a fixed list size is reported because adviser time, not a probability threshold, sets the list length.