
← ResearchResearch note11 min read
What sells together
Every bundle a shopper builds is a small decision about what belongs together. Most stores throw that decision away.
Abstract
When a shopper builds a bundle they reveal, with their own money, which products they think belong together, and a standard order report discards that information. Three decades of published work on market baskets show how to read it: count how often products appear together, then correct for how popular each one is on its own. The same literature is frank about the traps, from famous stories that were never quite true to pairings that are coincidence and recommendation figures that overstate their own effect. For a bundle, the practical lesson is to ask which pairings are doing the work, and to keep asking, because the answer changes through the year.
Key findings
- 01Raw counts of what is bought together mostly rediscover your bestsellers; the useful measure compares a pairing with what chance alone would produce.
- 02The "beer and diapers" story traces back to a real 1992 analysis, but the store never moved the shelves and even the origin is disputed.
- 03Products can share a basket through coincidence as well as through fit, and the classic grocery studies found true cross-category effects to be small.
- 04The claim that 35% of Amazon purchases come from recommendations is a single unsourced sentence in a 2013 consulting article, and careful studies find much smaller causal effects.
- 05Pairings are seasonal, so a picture of what sells together has a date on it.
When a shopper builds a bundle, they are answering a question no survey can ask: given these options, what belongs together? They answer it with their own money, one product at a time, and most stores throw the answer away.
An order report will tell you what was sold. It flattens the bundle into line items, and the decision that produced it disappears. You can see that the dripper sold well. You cannot see that it sold well because of what it was sitting next to.
Economists call this kind of evidence revealed preference: what people do when the choice is real, as opposed to what they say they would do. A build-a-box page produces it in an unusually clean form, because every order is a set of products one person chose to put side by side. This note looks at what the published research says about reading sets like that, where it has gone wrong, and what it means for anyone deciding what goes in a bundle.
How baskets became a science
The modern study of what sells together starts in 1993, at IBM’s Almaden Research Center. Bar-code scanners had made it possible, for the first time, to store what was in each individual basket and not just the day’s totals. Rakesh Agrawal, Tomasz Imielinski and Arun Swami asked what could be learned from a large database of those baskets, and proposed looking for association rules: statements of the form "customers who buy this also tend to buy that".1 The decisions they had in mind were ordinary retail ones: what to put on sale, how to design coupons, how to place products on shelves.
A year later Agrawal and Ramakrishnan Srikant published the algorithm that made the idea practical, known as Apriori.2 The difficulty is scale. A store with a thousand products has roughly half a million possible pairs and far more possible sets of three or four, so checking all of them against millions of baskets is hopeless. Apriori rests on one observation: a set of products can only be common if every smaller set inside it is common too. If filter papers and a scale are rarely bought together, there is no need to count how often filter papers, a scale and a kettle are. The 1994 paper reported speed-ups ranging from a factor of three on small problems to more than ten on large ones, and its example of a rule has been quoted ever since: that 98% of customers who purchase tyres and car accessories also have servicing done.2
Those two papers also fixed the vocabulary. Every rule was described by two numbers, support and confidence. A third, lift, arrived a few years later for reasons we come to below. All three are simple fractions, and they are easier to follow with a bundle in front of you.
Why counting is not enough
Support and confidence have a flaw, and it is the first thing a merchant runs into on opening a "frequently bought together" report: the top of the list is the bestsellers. A popular product appears beside everything, because it appears in most orders. That is a fact about its popularity and tells you nothing about what it belongs with.
The clearest statement of the problem came in 1997 from three Stanford researchers, Sergey Brin, Rajeev Motwani and Craig Silverstein, a year before Brin co-founded Google.3 Their example is a grocery store in which 25% of baskets contain tea, 90% contain coffee and 20% contain both. The rule "people who buy tea also buy coffee" looks excellent: support of 20% and confidence of 80%. But 90% of all shoppers buy coffee. A tea buyer is therefore less likely to buy coffee than a shopper picked at random. The two products are, mildly, substitutes, and the rule points the wrong way. In the authors’ words, stating it alone is "at best incomplete information and at worst misleading".3
Their fix is the ratio in the callout above: divide how often the pair really occurs by how often it would occur if the two purchases were unrelated. For tea and coffee that is 0.20 ÷ (0.25 × 0.90) = 0.89, below 1. They called the measure interest; it is now usually called lift. They also proposed using the chi-squared test, a standard statistical check, to judge whether a departure from 1 is large enough to be more than noise.3
Run the same check on the invented coffee shop. Suppose the house blend is in 900 of the 1,000 boxes, and in 170 of the 200 dripper boxes. Dripper and house blend is the most frequent pairing on the store, with 85% confidence. Its lift is 0.85 ÷ 0.90, or 0.94. The dripper does nothing for the house blend. The pairing that carries the box is the quieter one, dripper and filter papers.
Beer, diapers and a story that grew
No account of this field is complete without its legend. A big retailer, usually said to be Wal-Mart, mines its tills and discovers that young fathers sent out for diapers (nappies, on this side of the Atlantic) on a Friday evening pick up beer as well. The store moves the beer next to the diapers and sales of both jump. By 1997 the pairing was already familiar enough for Brin and his co-authors to call it "a classic example" in a footnote.3
The most careful tracing is by Daniel Power, an information systems professor who runs the DSSResources archive, as later summarised in The Register.4 In his account the story goes back to 1992, when Thomas Blischok, who led a retail consulting group at Teradata, analysed 1.2 million baskets from Osco Drug stores. The analysis did find that between 5pm and 7pm shoppers bought beer and diapers. It established nothing about the age or sex of those shoppers, and Osco does not appear to have rearranged a single shelf. A second account, from the analyst Mark Madsen, credits a different Teradata consultant, Karen Heath, working with plain database queries for an unnamed Midwest retailer in the same year, and adds a twist: the only shelf move he could document was by a drugstore chain that read about the correlation in a trade magazine in 1997 and acted on the article.5
So the famous example of data-driven merchandising is a modest finding from simple queries, with a disputed author, that the original retailer did not act on. The details that make it memorable were added in the retelling. We find that a useful thing to keep in mind whenever a pairing comes with a good story attached.
Bought together is not the same as belongs together
Suppose the beer and diapers pattern was real. What would it mean? In 1999 Puneet Manchanda, Asim Ansari and Sunil Gupta gave the most useful answer, in a paper that uses that very pair as its example.6 They separated three reasons two categories turn up in one basket. The first is complementarity: one product makes the other more worth having, so a promotion on one raises sales of the other. The second they called co-incidence: the products are bought on the same trip for reasons that have nothing to do with each other, such as similar purchase cycles. The third is the household itself: a large family buys more of everything.
The distinction matters because only the first is something a merchant can use. A discount on one true complement lifts its partner. A discount on a coincidental partner does nothing. Fitting their model to household purchases of cake mix, frosting, detergent and fabric softener, the authors found that the effect of one category’s price and promotion on another was real but generally smaller than its effect on itself, and often lopsided, with one product in a pair leading and the other following.6
A year later Gary Russell and Ann Petersen built a model in which any pair of categories could come out as complements, substitutes or unrelated. Across four grocery categories they found the cross-category price effects to be small, and concluded that the sheer flow of shoppers through the store may matter more than relationships between products in predicting what ends up in the basket.7
That is a sobering result, and we think the right reading is about the setting. A supermarket basket is assembled from tens of thousands of products for dozens of unrelated reasons, so most of what shares a basket is coincidence. A bundle is a different object. The shopper has been asked a narrower question, on one page, about one small set of products meant to be used together. The signal is cleaner. It is still not free of coincidence, and the tests above still apply.
Customers who bought this
The idea reached most shoppers through Amazon. In 2003 Greg Linden, Brent Smith and Jeremy York described the method behind its recommendations, which they called item-to-item collaborative filtering.8 Earlier systems looked for customers similar to you and recommended what they had bought, which becomes impossibly slow with tens of millions of customers. Amazon turned the problem round: for each product, work out ahead of time which other products tend to be bought by the same customers, and store that as a table. Recommending then means looking up a shopper’s few items in the table, which is fast however large the catalogue is and works from as few as two or three purchases.
How much is that worth? The figure quoted everywhere is that 35% of what consumers buy on Amazon comes from recommendations. It is worth knowing where it comes from. The 2003 paper contains no such number. The figure appears in a 2013 McKinsey article on retail, in one sentence alongside a similar claim that 75% of Netflix viewing comes from recommendations.9 We could find no method, sample or Amazon source behind it. It has been repeated for more than a decade as though it were a measurement.
Researchers who have measured the question find something smaller. Amit Sharma, Jake Hofman and Duncan Watts studied the browsing of 2.1 million Amazon users and estimated that at least 75% of the clicks on recommendations would have happened anyway, through search or direct browsing.10 The recommendation was pointing at where the shopper was already going. Anuj Kumar and Kartik Hosanagar ran a controlled experiment on a fashion retailer’s site and found that showing recommendations raised overall product sales by about 11%: a real effect, and a long way from a third of revenue.11
| Study | Setting | Finding |
|---|---|---|
| Brin, Motwani and Silverstein (1997)3 | Worked grocery example | A rule with 80% confidence can describe a negative relationship (lift 0.89) |
| Manchanda, Ansari and Gupta (1999)6 | Household panel, four grocery categories | Cross-category effects are smaller than own-category effects and often asymmetric |
| Russell and Petersen (2000)7 | Four grocery categories | Cross-category price effects are small; store traffic may matter more |
| McKinsey (2013)9 | Amazon, no method given | 35% of purchases attributed to recommendations |
| Sharma, Hofman and Watts (2015)10 | 2.1 million Amazon users, nine months | At least 75% of recommendation click-throughs would probably have happened anyway |
| Kumar and Hosanagar (reported 2018)11 | Experiment on a fashion retail site | Recommendations raised overall product sales by about 11% |
Pairings move with the calendar
Everything above treats a store’s orders as one pile. They are not. As early as 1998 Banu Özden, Sridhar Ramaswamy and Abraham Silberschatz pointed out that a rule can hold strongly in some periods and vanish in others, and that many do so on a regular cycle: by hour of day, day of week or month of year.12 A pairing that is invisible in a year of pooled data may be the strongest thing on the store for six weeks.
The products that sell together in March are not the ones that sell together in November.
For bundles this is close to the whole point. Gift boxes, seasonal flavours and back-to-school kits are built on pairings that are true for a while. A picture of what sells together is only useful with a date on it, and more useful still when you can watch it change.
What is not known
The literature leaves real gaps, and they bear directly on bundles.
- •Small numbers. The statistical tests behind lift were designed for large retailers. Brin and his co-authors note that their test becomes unreliable when the expected number of baskets holding a pair is small.3 A store with a few hundred bundle orders and forty products has many pairs seen only once or twice, and a lift calculated from two orders is noise.
- •Cause. Almost all of this work describes what happened. Whether putting two products together would make more people buy them is a different question, and the recommendation studies show how far apart the two answers can be.10 A fair test is the only way to settle it.
- •The menu. Shoppers can only pair what they are shown. A bundle with six options says nothing about the seventh product that was never offered, and the order of products on the page shapes what gets picked.
- •Bundles themselves. The classic studies are of supermarket baskets and large marketplaces. We know of far less published work on choices made inside a build-your-own bundle, where the set is small and the shopper is choosing on purpose.
What it means for a bundle
The question to ask of any bundle is which products are doing the work and which are along for the ride. Some pairings are the reason the bundle is bought at all. Others only make the page longer. The research gives that question a discipline: do not trust the raw count, compare against chance, be wary of small numbers and good stories, and look again when the season turns.
Knowing which is which changes how you merchandise. The pairing that carries the bundle goes to the front. The product nobody picks steps back or leaves. And sometimes the strongest pairing involves a product that is not in the bundle yet.
Kitenzo keeps the bundle whole instead of flattening it into line items. Merchants on Enterprise can see which products are chosen together across every bundle on their store, how strongly, and how that shifts over time. We are deliberately not going to describe how that picture is drawn. And because a pairing is a hypothesis until it has been tried, any merchant can put two versions of a bundle side by side in an A/B test and see which one sells.
Today the picture is something you read. The next step is to bring it into the bundle itself: a suggestion, at the moment it is useful, that a product belongs on the path. That work is still in the lab, and is not in the app.
References
- 1Agrawal, R., Imielinski, T. and Swami, A. (1993). Mining Association Rules between Sets of Items in Large Databases. Proceedings of the 1993 ACM SIGMOD Conference, 207–216.
- 2Agrawal, R. and Srikant, R. (1994). Fast Algorithms for Mining Association Rules. Proceedings of the 20th VLDB Conference, 487–499.
- 3Brin, S., Motwani, R. and Silverstein, C. (1997). Beyond Market Baskets: Generalizing Association Rules to Correlations. Proceedings of the 1997 ACM SIGMOD Conference, 265–276.
- 4Whitehorn, M. (2006). The parable of the beer and diapers. The Register, 15 August 2006 (summarising the tracing by D. J. Power, DSSResources, 2002).
- 5Swoyer, S. (2016). Beer and Diapers: The Impossible Correlation. TDWI Upside, 15 November 2016.
- 6Manchanda, P., Ansari, A. and Gupta, S. (1999). The "Shopping Basket": A Model for Multicategory Purchase Incidence Decisions. Marketing Science, 18(2), 95–114.
- 7Russell, G. J. and Petersen, A. (2000). Analysis of Cross Category Dependence in Market Basket Selection. Journal of Retailing, 76(3), 367–392.
- 8Linden, G., Smith, B. and York, J. (2003). Amazon.com Recommendations: Item-to-Item Collaborative Filtering. IEEE Internet Computing, 7(1), 76–80.
- 9MacKenzie, I., Meyer, C. and Noble, S. (2013). How retailers can keep up with consumers. McKinsey & Company.
- 10Sharma, A., Hofman, J. M. and Watts, D. J. (2015). Estimating the Causal Impact of Recommendation Systems from Observational Data. Proceedings of the 16th ACM Conference on Economics and Computation (arXiv:1510.05569).
- 11University of Florida, Warrington College of Business (2018). How valuable are online product recommendations? (report of Kumar, A. and Hosanagar, K., Information Systems Research). University of Florida.
- 12Özden, B., Ramaswamy, S. and Silberschatz, A. (1998). Cyclic Association Rules. Proceedings of the 14th International Conference on Data Engineering, 412–421.


