Insurance
The complaints that will be upheld, at the top of the queue on the day they arrive
An intake score for an insurer’s complaints team, tested on 64,462 real complaints to a state insurance regulator against working the queue in arrival order.
Executive summary
An intake score puts 45% of regulator-confirmed complaints in the top 30% of an insurer’s queue, against 36% for a reason-based rule, tested on 64,462 later complaints.
Situation
An insurer’s complaints team works its queue in arrival order, so the complaints most likely to be upheld wait behind routine ones.
Approach
An intake score from reason, line of business, complainant role and the insurer’s record, fitted on 115,785 real complaints and tested on 64,462 later ones.
Result
The top 30% of the queue holds 45% of confirmed complaints; the top tenth is confirmed 55% of the time.
The problem
An insurer’s internal dispute resolution team works complaints in arrival order against RG 271 time limits. Some of those complaints will end up upheld by AFCA or the regulator, and finding them on day one lets a senior case manager fix them before they escalate. AFCA’s Datacube and ASIC’s IDR dashboard show how outcomes vary by product and issue.
I measured how many of the complaints that would be upheld an intake score puts at the top of the queue on the day they arrive, against working in arrival order.
- Team
- Head of complaints, internal dispute resolution handlers, claims and underwriting leads
- Scale
- Thousands of complaints a year across motor, home, health and life
- Systems
- Salesforce Service Cloud or Dynamics 365, Guidewire or Duck Creek for claims
- Measure
- Confirmed complaints reached in the first review pass; days to senior review
What I built
An intake score that rates each new complaint for the chance it will be confirmed against the insurer, using only fields captured on arrival. The queue is sorted by that score, so a senior handler opens the highest-risk complaints first and the routine ones flow to the general queue. A second score rates the chance the customer ends up receiving more money, which flags complaints worth settling early.
Results
I tested on real complaints made to the Texas Department of Insurance: 180,247 closed complaint lines against insurers from 2018 to 2026, each with the regulator’s finding. The score was fitted on complaints received before 2024 and tested on the 64,462 received from January 2024 onward.
In the top tenth of the queue, 55% of complaints were confirmed, almost twice the 28% average. The score separates confirmed from unconfirmed complaints with an AUC of 0.66; the reason-based rule manages 0.58 and the insurer’s past record alone 0.58.
| Share of queue reviewed first | Confirmed complaints found: intake score | Reason-based rule |
|---|---|---|
| 10% | 19% | 14% |
| 20% | 33% | 25% |
| 30% | 45% | 36% |
| 40% | 56% | 47% |
| 50% | 66% | 59% |
Which customers get more money
The second score, for complaints that end with the customer receiving additional money, reaches an AUC of 0.75. 24% of confirmed complaints ended in an extra payment, against 9% of the rest. These are the complaints where an early, fair offer shortens the dispute.
When the base rate moves
The share of complaints confirmed against insurers sat between 12% and 15% from 2018 to 2022, reached 19% in 2023 and has run at about 28% since 2024. A model fitted on the earlier years under-predicts the level, while its ranking still holds. I check the level each month and refit the calibration when it drifts, so the queue order and the forecast workload both stay right.
Data and method
- Data
- Texas Department of Insurance complaints against insurance companies, published on data.texas.gov: 180,247 closed complaint lines received 2018-01-02 to 2026-11-06, with reason, line of business, complainant role, the insurer and the regulator’s finding. No complainant is identified in the data or here.
- Target
- Complaint confirmed by the regulator; separately, additional money received by the complainant
- Split
- Fitted on complaints received before January 2024; tested on January 2024 to November 2026
- Methods
- Logistic regression on complaint reasons, line of business, coverage level, complainant role and type, and a smoothed confirmed rate for the insurer from the fitting years only; compared with arrival order and a reason-based rule
- Measures
- Confirmed complaints reached by share of queue reviewed; AUC; calibration by decile
Limitations
- A regulator’s complaint file stands in for an insurer’s internal register. An insurer’s own data adds policy, claim and contact history, which should strengthen the score.
- The finding is the regulator’s judgement, and its standard shifted in 2024; the monthly level check exists for that reason.
- The score orders the queue; it does not decide outcomes. Every complaint is still handled against the insurer’s internal dispute resolution standard and response timeframes.
- Insurers are not named here.
How I would take it live
- Export two years of closed complaints with outcomes from the complaints register
- Fit the intake score and the payment score, and agree the senior-queue share with the head of complaints
- Run in shadow for a month: handlers work as normal while the score is recorded and compared with outcomes
- Sort the live queue by score in Salesforce or Dynamics, with a monthly check of calibration and outcomes