Retail and e-commerce · Win-back
Win-back emails sent to the customers most likely to return
Monthly customer segments and a ranked win-back list for an online store, tested on a later period of 1,067,371 real order lines.
Executive summary
Win-back emails chosen by a model reach 284 returning buyers from 500 sends, against 259 by recency and about 145 at random, on two years of real store orders.
Situation
An online store sends the same newsletter to everyone and its win-back flow to whoever bought last. Discounts leak margin to customers who would reorder anyway.
Approach
Four plain-named segments and a 90-day reorder score from order history, chosen on an earlier window and tested once on customers as at September 2011.
Result
The score beats recency modestly and reliably. Gradient boosting lost to logistic regression and was not used. Segments set the message; the score sets who gets it.
The business problem
An online store’s win-back emails usually go to everyone who has not bought in 90 or 180 days. Retailers using flows in Klaviyo or similar report that win-back works best when the timing and offer follow each customer’s own buying gap and order value, rather than one date for all.
I measured how many returning buyers a ranked monthly win-back list reaches per 500 emails against a recency rule and against sending at random.
- Team
- Owner plus one marketing or e-commerce coordinator
- Scale
- A few thousand customers, one to four campaigns a month
- Systems
- Shopify or WooCommerce orders, Klaviyo or Mailchimp, Xero
- Constraint
- Spam Act 2003 consent and unsubscribe rules; Privacy Act for customer data
What was built
A monthly job that reads order history, places every customer in one of four plain-named segments, and scores each lapsed customer's chance of buying in the next 90 days. Segments and scores sync to the email tool as tags; the top of the lapsed list gets the win-back flow. A second score reads on-site behaviour (pages viewed, product-page time, exits) to flag sessions likely to convert, for remarketing.
Results
Four segments, four actions
| Segment | Customers | Share of revenue | Bought again in 90 days | Action |
|---|---|---|---|---|
| Loyal big spenders | 871 | 67.3% | 84.2% | Account-style contact; early access, no discounts |
| Drifting | 1,607 | 23.9% | 47.0% | Win-back email with a reason to return |
| Lapsed one-offs | 2,131 | 5.3% | 20.4% | Low-frequency newsletter only |
| Regulars | 647 | 3.5% | 56.3% | Replenishment reminders |
Who to email first
| Method | AUC (95% CI) | PR-AUC |
|---|---|---|
| Rule: bought in last 90 days | 0.688 (0.675–0.701) | 0.581 |
| Rule: recency rank | 0.762 (0.749–0.775) | 0.685 |
| Logistic regression | 0.789 (0.778–0.801) | 0.761 |
| Gradient boosting | 0.761 (0.748–0.774) | 0.740 |
The model was chosen on an earlier window and tested once on a later one. Logistic regression beat recency ranking by 0.027 AUC (95% CI 0.018 to 0.035); gradient boosting lost to it and was not used.
On-site buying signals
Data and method
- Orders
- UCI Online Retail II (Chen, 2019), CC BY 4.0: 1,067,371 invoice lines, 5,942 customers, 43 countries, Dec 2009 to Dec 2011
- Cleaning
- 19,494 cancellations, 34,335 duplicates and 5,991 service lines removed; 22.8% of rows have no customer ID and are left out of customer models
- Split
- Select on a window to 2011-03-12; test once on customers as at 2011-09-10, outcomes to 2011-12-09
- Sessions
- UCI Online Shoppers Purchasing Intention (Sakar et al., 2018), 12,330 sessions
Segments come from k-means on log recency, frequency and spend (four clusters; silhouette checked from two to eight). The 90-day model uses recency, frequency, spend, basket size, tenure and last-90-day activity.
Assumptions and limitations
- Prediction is not persuasion: some of the 284 would have returned anyway. A hold-out group is needed to measure the lift from the email itself
- The spend forecast ran far below actual in the test quarter (£1.46M against £2.81M, because of the pre-Christmas peak); it is used only to rank, where its top 10% still capture 60% of next-quarter revenue
- 54% of buyers the model missed were one-time customers with a median 304 days since their order
- A UK gift wholesaler selling to shops; a consumer store needs its own refit
What a store would do next
- Week 1: export 24 months of Shopify orders and customers
- Week 2: segment, score and check against last quarter; agree messages per segment
- Weeks 3 to 8: run the win-back flow with a 20% hold-out
- Then: keep or stop on reorders and margin against the hold-out
Technical depth
Out-of-time testing matters here because seasonality is strong: a random split would mix November into training and testing. Guest checkouts are reported separately so the share of "known" customers is not overstated.