Business

You Probably Use Online Reviews Wrong

You're on a road trip and getting hungry, so you decide to check restaurant reviews for the next town you'll drive through. Slim pickings: there are only two spots that sound the least bit appetizing. One has two stars with 500 reviews, the other has two stars with only 25 reviews. Which is more likely to be a good experience? We'll tell you right now: you probably chose wrong. According to research, most people do.

The Amazon Test

To examine how people use online reviews, researchers from Indiana University Bloomington and University of California, Los Angeles first had to prove what the best way to use them really was. They used a statistical model to find patterns in millions of reviews for products on Amazon. They found that there was no relationship between the number of reviews and the average rating — that is, a product with 5,000 reviews was just as likely to have two stars as to have five stars, and a product with five stars was just as likely to have 5,000 reviews as to have five.

Next, they recruited 132 people to look at pairs of phone cases, presented online the way they would on an actual online store. After checking out the average user rating and the total number of reviews, they were asked to select which of each pair they would buy. Across the board, the participants chose the product with more reviews.

What's wrong with that? The number of reviews isn't a sign of how good a product is; it's just an indicator of how accurate the average rating is. Statistically, a two-star product with 5,000 reviews is more likely to be of two-star quality than a two-star product with five reviews. That means that when faced with two low-rated products, you should choose the one with fewer reviews because that rating is more likely to be wrong. It's no guarantee, but the odds are in its favor.

What's Right Is Not Always Popular

The problem people get into is judging a product — or restaurant, or hotel — by its popularity, not its rating. But plenty of things that are popular aren't necessarily good (Smoking! Sleep deprivation! Smooth jazz!). As the motherly adage goes, if all your friends jumped off a bridge, would you?

We tend to go along with the pack because of a concept called social proof. If we see all our friends driving Ford Pintos, we feel like we'd be missing out if we didn't buy a Ford Pinto. (Protip: Don't drive a Ford Pinto unless you want to burst into flame in a rear-end collision.)

According to the researchers, this study says we all have a lot of learning to do. "Consumers try to use information about other people's experiences to make good choices, and retailers have an incentive to steer consumers toward products they will be satisfied with," says lead author Derek Powell of Stanford University. "Our data suggest that retailers might need to rethink how reviews are presented and consumers might need to do more to educate themselves about how to use reviews to guide their choices."

Statistics Basics

Written By Ashley Hamer September 16, 2017