Manaz Afzal

Manufacturing · After-sales service

Which trucks get an air-system check, from data they already log

A workshop score built on Scania’s published data from 76,000 heavy trucks in everyday use, with the cut-off set from what a check and a breakdown cost.

365 of 375failures caught, checking 4.8% of trucks
8,960test cost; best single-sensor rule 19,640, check every truck 156,250
98%of failures in the top 5% of scores

Executive summary

A score from readings trucks already log sends 4.8% of trucks for an air-system check and catches 365 of 375 failures, at under half the cost of the best single-sensor rule.

365 of 375air-system failures caught by checking 761 of 16,000 test trucks
8,960total test cost on Scania’s published 10:500 scale; single-sensor rule 19,640
98%of failures sit in the top 5% of scores

Situation

Checking every truck’s air system wastes technician hours; checking none leaves brake and gear-change failures on the road. The workshop needs to know which trucks to inspect.

Approach

LightGBM on 170 on-board readings from 60,000 Scania trucks, with the cut-off set from what a check and a missed failure cost, chosen on training data and applied once to 16,000 test trucks.

Result

Cost 8,960 (95% CI 6,170 to 12,201), matching the best published result on this data (9,920). The score stays the cheapest option from 10:1 to 200:1 cost ratios.

The business problem

A truck dealer’s service network or a large fleet workshop sees trucks come in for service with faults that are not yet obvious. The air pressure system supplies the brakes and gear changes, and a failing component that leaves the workshop unchecked can mean a roadside breakdown, a tow and a missed delivery. Manufacturers already use connected-truck data to tailor service plans, and the question for the workshop is which trucks to inspect, from counters the truck already logs.

I set the cut-off from what an inspection costs against a breakdown and measured how many failures a score catches for the checks a workshop can afford.

Team
A service manager, 4 to 15 heavy-vehicle technicians, a parts desk
Fleet
300 to 20,000 trucks reporting to the service network
Systems
Telematics or on-board counters read at service, a dealer management system, workshop job cards
Constraint
Heavy Vehicle National Law brake standards; a technician makes the call, the score sets the order

What was built

A score for each truck from its on-board counters and histograms, read when the truck is connected at service or through telematics. Trucks above a cut-off get an air-system inspection added to the job card. The cut-off is set from what a check and a missed failure cost, not from accuracy.

Truck connectedOn-board countersand histogramsScoreRisk the air-systemcomponent is failingCut-offSet from cost of acheck against a missJob cardInspection addedfor flagged trucksOutcomeFound, not found;logged for refit
Workshop workflow. A technician confirms every flagged truck before parts are ordered.

Results

Scania published this data with a cost: 10 for an unnecessary check by a mechanic, 500 for a missed failing truck. The test set holds 16,000 trucks, 375 of them with the failing component.

Check every truck16,000 checks156,250Check no trucks375 missed187,500Single-sensor rule1,421 checks, 18 missed19,640Logistic regression1,017 checks, 13 missed13,050Gradient boosting761 checks, 10 missed8,960
The score sends 761 trucks for a check (4.8%) and catches 365 of 375 failures. Total cost 8,960, against 19,640 for the best single-sensor rule and 156,250 for checking every truck. Data: APS Failure at Scania Trucks, UCI 421; 60,000 training and 16,000 test trucks
ApproachChecksFailures caughtMissedUnneeded checksTotal cost (95% CI)
Check every truck16,000375 of 375015,625156,250 (155,900–156,620)
Check no trucks00 of 3753750187,500 (168,988–205,000)
Single-sensor rule1,421357 of 375181,06419,640 (15,950–24,233)
Logistic regression1,017362 of 3751365513,050 (9,730–17,040)
Gradient boosting761365 of 375103968,960 (6,170–12,201)

The cut-off was chosen on training data only and applied to the test set once. The best published entry in the 2016 industrial challenge on this data scored 9,920 (542 unneeded checks, 9 missed). This pipeline scores 8,960 (396 unneeded, 10 missed); the 95% interval (6,170 to 12,201) overlaps it, so the fair reading is that it matches the best known result. Against logistic regression the saving is 1,319 to 7,050 (95% CI).

010,00020,00030,0004%8%12%16%20%Chosen on training data:4.8% checked, cost 8,960Share of trucks sent for an air-system check
The cost curve is flat near its lowest point, so the cut-off does not need to be exact. Checking fewer than 4% of trucks lets failures through quickly; checking more than 8% wastes hours.

Where the failures sit

Top 5%366 of 800 trucks98%Top 6–10%7 of 800 trucks2%Top 11–15%0 of 800 trucks0%Top 16–20%1 of 800 trucks0%Top 21–25%0 of 800 trucks0%Top 26–30%1 of 800 trucks0%
The top 5% of scores (800 trucks) hold 366 of the 375 failures (98%). Below the top 10%, the score finds 2 failures in 14,400 trucks.

What the score reads

0 missing591 trucks12.9%1–5 missing4,371 trucks3.2%6–10 missing38,121 trucks0.1%11–50 missing13,995 trucks3.3%51+ missing2,922 trucks9.5%All 1.7%
Missing readings carry signal. Failing trucks have a median of 20 missing values against 8 for others, and trucks with no gaps at all fail at 13%. A gap may mean a counter that stopped logging; the model is allowed to see that, and the data team should check why.

Sensor names are anonymised by Scania, so the score cannot say which physical reading matters. One counter, ck_000, carries a third of the model’s gain and is also the best single-sensor rule. In a live service network the names are known, and the reasons can go on the job card.

Data and method

Source
APS Failure at Scania Trucks, Scania CV AB (2016), UCI ML Repository 421, GPL-3
Size
76,000 trucks in everyday use; 170 anonymised readings, 7 of them histograms with 10 bins
Split
Published split: 60,000 train (1,000 failures), 16,000 test (375 failures)
Audit
8.3% of readings missing; 99% of trucks have at least one gap; 8 readings over half missing; 1 constant reading

Four approaches beyond check-all and check-none: the best single reading and threshold found on training data, logistic regression on log-scaled readings with a missing count, and LightGBM (385 rounds). Cut-offs come from five-fold out-of-fold predictions on the training set. Cost intervals from 1,000 bootstrap redraws of the test trucks.

Every truck in the data had a fault of some kind. The negative class is trucks whose fault was not in the air-system component. The score separates the two; it was not tested on healthy trucks.

Assumptions and limitations

If the costs are different

Cost of a miss against a checkModelCheck every truckCheck noneChecksFailures caught
10:15,100156,25037,500500340 of 375
20:16,180156,25075,000636358 of 375
50:18,960156,250187,500761365 of 375
100:110,190156,250375,0001,192373 of 375
200:112,190156,250750,0001,192373 of 375

Cut-off re-chosen on training data at each ratio, then applied to the test set.

What a service network would do next

Technical depth

Missing values are left as missing for gradient boosting, which learns its own split direction for them; logistic regression gets median fill, log scaling and a missing count. The positive class is weighted five times in boosting, then the cut-off is set on expected cost, which matters more than the weight. The single-sensor rule is searched over every reading and 100 thresholds per reading.

Analysis code (Python, LightGBM, scikit-learn)

Financial impact

About A$16k a year saved against the single-sensor rule at 2,000 service visits

Costs are converted to an Australian workshop at the 50:1 ratio used in the study, so the trade-off between checks and misses is unchanged.

InputValueSource
Truck service visits a year2,000 dealer workshop
Cost of an air-system checkA$120 about one technician hour
Cost of a missed failureA$6,000 tow, repair premium, missed delivery
Checks and misses per 16,000: model761 checks, 10 missedMeasured
Checks and misses per 16,000: single-sensor rule1,421 checks, 18 missedMeasured
Failure rate in the test set2.3%Measured 375 of 16,000

Calculation

  1. Annual cost: check every truckA$240,000
  2. Annual cost: single-sensor ruleA$34,815
  3. Annual cost: modelA$18,915
  4. Saving against the ruleA$15,900

Annual saving against the best single-sensor ruleA$15,900

Safety value of fewer roadside brake failures is not priced. Every truck in the data had some fault, so the score was not tested on healthy trucks.

Implementation and rollout

Five weeks to a refit with named readings, then a three-month branch comparison

  1. Weeks 1 to 3

    Export

    18 months of on-board readings and air-system job outcomes from the dealer management system.

    Service systems lead, Manaz

  2. Weeks 4 to 5

    Refit and set the cut-off

    Refit with named readings, set the cut-off from the workshop’s own costs, agree job-card wording.

    Service manager, Manaz

  3. Months 2 to 4

    Branch comparison

    Flagged inspections at half the branches; compare roadside breakdowns and technician hours.

    Service managers

  4. Then

    Extend

    Other systems that log the same counters; quarterly refit as trucks age.

    Service network lead

Tools

Telematics or on-board counters read at service, Dealer management system and job cards, Python, LightGBM and scikit-learn, Job-card rule in the DMS

Team

  • Service manager owns the cut-off and the trial
  • Technicians confirm every flagged truck before parts are ordered
  • Manaz model, cost curve and integration

The timeline is a plan for a typical engagement; it depends on data access and how many systems are involved.

Risks and mitigations

What could go wrong in production, and the control for each

RiskMitigation
A failing truck scores low and leaves uncheckedScore adds checks; it never removes brake inspections required under Heavy Vehicle National Law.
Readings missing because a counter stopped loggingMissing counts are a signal here; a weekly data report flags counters that go silent.
Snapshot data hides ageing effectsRefit on readings over time and check the score holds as trucks age.
Costs differ by workshopCut-off re-chosen from local labour and breakdown costs; the cost curve is flat near its minimum.

Key takeaways

Key takeaways

  • Set the cut-off from what a check and a miss cost, not from accuracy.
  • The score matches the best published result on this benchmark, and the bootstrap interval is reported, not a single number.
  • Missing readings carry signal: failing trucks have a median of 20 gaps against 8.
  • The model adds inspections in a set order; a technician makes every call.