A pricing review that shows which motor customers the rating table overcharges, and which it undercharges
A yearly refit of claim frequency on 678,013 real motor policies, compared with a conventional rating table and tested on policies the models never saw.
Executive summary
A boosted frequency model finds where a motor rating table charges good drivers double and risky drivers half, on 678,013 real policies, without changing the total premium.
Situation
A small insurer or underwriting agency reviews motor rates once a year with a factor-by-factor rating table. Combinations the table does not see are mispriced: good customers leave at renewal, risky ones stay.
Approach
Claim frequency refitted on 678,013 French motor policies with a Poisson GLM (the rating table) and LightGBM, on a group split so near-identical policies never sit on both sides. 135,920 test policies.
Result
All methods predict the 7,149 claims in total; the gain is in who pays. Deviance falls 8.1% from a flat rate with the boosted model against 4.9% for the table. 73% of policies would move more than 10%, so changes must be capped and phased.
The business problem
A motor insurer prices from a rating table built on factors such as driver age, vehicle and area, set years ago and adjusted in small steps. Pricing teams review it with GLMs of claim frequency and severity, and those reviews often find factors that are weak or out of line, which means some customers subsidise others.
I measured where the rating table and a model of expected claims disagree most, and how much the table overcharges good drivers and undercharges risky ones in those cells.
- Team
- One pricing analyst, an underwriting manager and a claims lead
- Book
- 20,000 to 700,000 policies; claims in about 1 in 10 policy-years
- Systems
- Policy admin system export (Guidewire, Duck Creek or in-house), Excel rating tables, a claims system
- Constraint
- General Insurance Code of Practice, the Privacy Act and pricing that can be explained to a customer; the model proposes, the underwriter signs off
What was built
A yearly pricing review that refits claim frequency on every policy-year, compares the rating table with a boosted model, and lists where they disagree. The underwriter gets the factor tables, a list of segments the table misprices, and the price change each segment would see. Nothing changes in the rating engine until it is signed off.
Results
Both ends cost money. The first group are good customers paying double; they are the ones a cheaper competitor wins at renewal. The second group pay about half their expected claims.
| Method | Mean deviance (lower is better) | Gini | Claims predicted | Claims seen |
|---|---|---|---|---|
| Flat rate | 0.3278 (0.3219–0.3334) | 0.066 | 7,226 | 7,149 |
| Rating table (Poisson GLM) | 0.3118 (0.3064–0.3169) | 0.279 | 7,235 | 7,149 |
| Boosted model (LightGBM, Poisson) | 0.3011 (0.2953–0.3066) | 0.332 | 7,225 | 7,149 |
All three methods predict close to the 7,149 claims seen in the test year, so the gain is in who pays, not in the total. Mean Poisson deviance falls 4.9% from flat to the table and 8.1% to the boosted model. The boosted model's gain over the table has a 95% interval of 0.0095 to 0.0119, clear of zero.
What the price change looks like
As a pure premium (frequency times capped average claim cost, no expenses or profit), the flat price is 171 euros. The table spreads the middle 80% of customers from 101 to 368; the boosted model from 80 to 330.
Factor by factor
| Rating factor | Share of model gain |
|---|---|
| Bonus-malus (claims record) | 25.2% |
| Vehicle age | 22.1% |
| Driver age | 11.7% |
| Vehicle make | 9.8% |
| Region | 9.2% |
| Vehicle power band | 8.5% |
| Population density | 8.3% |
| Fuel type | 3.8% |
| Area code | 1.4% |
Claim costs
Frequency is half the price. Claim costs in this book are dominated by a few large claims: the top 1% of 26,444 paid claims carry 38% of the cost, and the largest is 4,075,401 euros. The average claim is 2,266 euros; the median is 1,172.
The review caps claims at the 99.5th percentile (34,377 euros) and spreads the excess, 25% of all cost, across the book as a loading. An Australian book needs the same check for fixed amounts, such as excess-only claims or recovery settlements.
Data and method
- Source
- freMTPL2freq and freMTPL2sev, French motor third-party liability (CASdatasets), OpenML 41214 and 41215
- Size
- 678,013 policies, 358,499 policy-years, 36,102 claims; 26,444 claims with a cost
- Split
- Group split 80/20 on identical risk profiles: 542,093 train, 135,920 test, so near-identical policies never sit on both sides
- Audit
- 38% of rows share a profile with another row; 9,116 policies with claims have no cost record; 9 policies with 5+ claims capped at 4; 1,224 exposures over a year capped at 1
Three methods with exposure as the offset: a flat rate, a Poisson GLM on one-hot factors (the rating table), and LightGBM with a Poisson objective (943 rounds, early stopping on a held-out slice of train). Confidence intervals from 200 bootstrap redraws of the test policies.
A random row split, which lets near-duplicate policies sit on both sides, flatters the boosted model: it explains 8.7% of deviance that way against 8.1% on the group split. The figures above use the group split.
Assumptions and limitations
- French policies, mostly one year or less, from a single insurer; an Australian book has different factors (postcode, garaging, vehicle value, NCD) and must be refitted on its own history
- Pure premium uses capped average claim cost for every policy; a separate severity model by segment is the next step
- Third-party liability only, no own-damage or theft cover
- Retention is not modelled; how many overcharged customers would stay at a lower price needs the insurer’s renewal data
- Some factors (age, area) need a fairness and explainability review before use; the model only proposes changes
What an insurer would do next
- Weeks 1 to 2: export three years of policy-years, claims and costs from the policy admin and claims systems
- Weeks 3 to 4: refit, compare with the current table, review the largest disagreements with the underwriter
- Next renewal cycle: apply capped changes to one product or channel and hold the rest as a comparison
- Then: track loss ratio and retention by segment for 12 months
Technical depth
Deviance is the mean Poisson deviance per policy, weighted by exposure through the offset. The Gini is exposure-weighted, from the ordered Lorenz curve. The double-lift chart sorts test policies by the ratio of boosted to table prediction and compares each with claims seen in tenths. Driver-age partial dependence and factor A/E tables are in the analysis output.