Do supermarkets know more about us than we do?

When you buy something at the supermarket, what makes you pick one brand - or one product - over another?

Retailers invest huge sums trying to understand this decision-making process, or influence it, in a UK grocery market worth almost £200bn.

We know supermarkets can learn a huge amount about us by studying what we buy.

But our research also suggests that we form personal bonds with products over time, as we try to self-justify our buying decisions.

In short, we do not just buy what we prefer, we end up preferring what we buy; and this means we are more open to influence than we might think.

Analysing the decisions we make in the supermarket can help us understand the choices we make in other areas of our lives.

Shaped by our choices

To explore shoppers' decision-making, we collected more than a million till receipts from one of the UK's major supermarket chains over several years.

Unsurprisingly, we discovered that people's purchases reveal what they value and what their aims are. These range from very specific goals such as making a stir-fry, to general goals such as reducing their overall spending.

But our relationships with products go a step further, according to a second study involving loyalty card customers.

Analysing the buying decisions of 280,000 customers revealed that they fell into a self-reinforcing cycle, where they became more and more attached to a product.

These cycles tend to last for several consecutive store visits before the pattern is broken and the process starts again with a different brand.

Interestingly, when consumers break out of these self-reinforcing loops, they tend to do so across multiple products at a time.

For example, when switching their brand of coffee, they are more likely to change their brands of yoghurt and washing detergent as well.

Blind loyalty

Why does this loyalty build up?

Further analysis of the data ruled out simpler explanations, such as price or force of habit being responsible for these patterns.

One explanation is that people come to like what they purchase, out of a need to "make sense" and explain their choices to themselves and others.

For example, after buying the ingredients for a salad, a consumer might start to value healthy foods more to justify the purchase.

This pattern of behaviour could be exploited to try to create a relationship with a selected product.

In the loyalty card study, we sent the supermarket's instant coffee drinking customers coupons to try a different brand.

Those in the switching phase were twice as likely to use the coupon as those still locked in to their existing coffee product.

Beyond shopping

This self-justified decision making is not limited to the weekly shop, but probably spills over to many areas of our lives.

For example, studies suggest people defend their selection of everything from the jam they buy to the politicians they decide to vote for in an election.

After we vote for a leader we may mimic their positions on many issues, including those we were undecided about or even to which we were opposed.

More stories like this

• The emotions that make a film a hit... or a miss

• Why the UK has such cheap food

• What Airbnb really does to a neighbourhood

Finding a 'hidden gem'

Human decision-making can sometimes be more logical, with takeaway food choices one example of this.

A wide-ranging study looked at users of a popular food delivery app in 197 cities, to deduce how they decided from which restaurant to order.

It is natural to think the app's users make their ordering decision based on ratings and popularity.

In reality, we found they were more interested in finding "hidden gems" with fewer positive reviews.

This may seem counter-intuitive and wrong, but this is actually how sophisticated machine-learning systems deal with uncertainty.

The more reviews a restaurant has, the more trustworthy the rating is.

When a restaurant only has a few reviews, even if they are not very positive, there is still a decent chance that the restaurant will be excellent. After all, those few reviewers may just be off the mark.

In contrast, when a restaurant has many good - but not excellent - reviews, the odds are vanishingly small that the restaurant is exceptional.

In effect, customers prefer to take a chance on greatness rather than a guarantee of mediocrity.

We found that people ordering in cities with better restaurants are more likely to take risks. This is a smart move, because you are more likely to chance upon an excellent restaurant in a city with a good food scene.

However, our research indicates that when people do explore and are burned by a bad experience, they return to familiar and unhealthy choices, like hamburgers.

A method to the madness

Although consumers might not be aware of it, their purchases reveal a great deal about them.

While they may appear to be mindlessly browsing food delivery options after a night out, the data trail they leave reveals a method to the madness.

We now know that when loyalty to one product is broken, consumers are likely to switch other products too. Shoppers could take advantage of this window to replace bad habits with good ones, but could also be more susceptible to advertisements for new products.

Our understanding of human decision-making is improving all the time. Combined with advancements in AI, this could help unlock not just our food choices but all of the decisions that we make.

About this piece

This analysis piece was commissioned by the BBC from experts working for an outside organisation.

Brad Love is a fellow at The Alan Turing Institute, the UK's national centre for data science and artificial intelligence.

He is also professor of Cognitive and Decision Sciences at University College London. You can follow him on Twitter here.

Professor Love has previously worked as a consultant for Customer Data Science company dunnhumby.

Edited by Eleanor Lawrie