Product reviews, in theory, are a very good idea. If you are buying a thing, why not consult the opinions of hundreds of people who have already bought and used that very thing? If you are trying to decide between three televisions, why not skim the reviews to see which is the best television? You deserve the best, after all! If a TV has hundreds of reviews saying it’s good, then it must be! Conversely, if it has no reviews, maybe no one has felt good enough about it to tell strangers on the internet it’s worth buying — a sign that you should steer clear.
But product reviews, much like the comments section of news articles, are mostly a cesspool of poorly formed, strongly held opinions. They’ll never actually help you determine which television is the absolute, objective best by reading reviews, but that doesn’t mean they don’t serve a specific — and useful — purpose. To get the most out of reviews, you can’t just rely on an aggregated star-ranking system. You actually have to read them.
As of this writing, more than 28,000 people on Amazon have reviewed the Instant Pot, the cult kitchen appliance that somehow slow-cooks stews and makes yogurt, though not at the same time. Nearly 18,000 people have used Amazon to review this Fitbit. Fenty’s Pro Filt’r foundation has more than 11,000 reviews on Sephora’s website.
The overwhelming majority of these reviews are relentlessly positive; these are four-star and higher products, per the rating system. But those reviewers who aren’t extolling the virtues of their fancy kitchen gadget or their foundation are instead explaining why it’s the very worst kitchen gadget or foundation to ever exist. (“All the features on this model are completely useless. … A lot of money laid out for zero return,” one disgruntled Instant Pot owner wrote. “By far the worst foundation I’ve ever bought. … Worst investment ever. Was not worth the hype,” wrote one of the 881 people who were disappointed with Rihanna’s foray into beauty products.)
In the world of online reviews, there is no middle ground; everything is either excellent or terrible, despite the fact that most things in this world are frankly just fine.
Instead of treating reviews as a sign that something is good or bad, the best or the absolute worst, as the one- to five-star system encourages us to do, we should instead treat reviews as what they are: guides to specific products’ pros and cons, as described by real people. Nothing more, nothing less.
The highs and lows (and ones and fives) of online reviews
Paul Pavlou, a professor of management information systems at the Fox School of Business at Temple University, told me the prevalence of both five- and one-star reviews is the result of a phenomenon called “underreporting bias.”
“We called it a J-shaped distribution in some of our earlier work, because there are more positive reviews than negative reviews,” Pavlou said. There are even fewer reviews in the middle. “It’s the extreme reviews — the ones and fives — that tend to be most common.”
There are two reasons for this, according to his research. The first is that people rarely buy something they expect to hate, so shoppers tend to be “positively disposed” toward the things they buy. The second is that writing a review is harder than not writing a review. “It’s extremely costly to write reviews, not in terms of monetary cost, but in terms of time and effort,” Pavlou said, “so you only do it if you have something to say.”
Even if most of these reviews are unhelpful, retailers are taking steps to show people the best reviews — the ones that will actually help them decide whether to buy a product, Ted Lappas, an assistant professor at the Stevens Institute of Technology’s School of Business, told me. Amazon, for example, highlights not only the “most helpful” good review but the “most helpful” bad review for most products. These are determined based on an algorithm that takes the review date into account, as well as feedback by other reviews (i.e., when you indicate that a review is helpful). “Stuffing the ballot box,” however, won’t affect the review’s rank, according to Amazon’s review guidelines.
The most helpful good review for the Instant Pot, for instance, is hundreds of words long and provides explanations for some of the problems people have with the appliance. The most helpful bad review, meanwhile, says that despite the Instant Pot being a great product, the manufacturer doesn’t honor the warranty. Sephora similarly lets shoppers sort reviews by “most helpful,” which pushes most one-line reviews to the bottom.
These algorithms aren’t just designed to weed out bad reviews — in some cases, they’re also designed to weed out bad products.
Amazon, by far the largest e-commerce website, takes the number and quality of reviews into account when sorting products. If you search for “headphones,” for example, you’ll get more than 20,000 results, but most people don’t bother to look past the first few pages. The products that get the coveted spots on those pages are a mix of sponsored products and those that have a high number of four- and five-star reviews.
The algorithm is designed to weed out bad products, but it can also bury new items from smaller companies, said Lappas. Most people only read a handful of reviews before deciding what to buy, he said, and very few will look past the first five or 10 items they see.
“We’re basically pushing customers to always buy the top few products,” Lappas said. “People never get to look at item six, or 16, or 20.” Products with lots of reviews are pushed to the top, encouraging more people to buy them and therefore begetting more reviews. Products with few reviews are in turn pushed to the bottom of the list, which prevents people from buying — and eventually reviewing — them, keeping them at the bottom of the list.
Brands that lack the advantage of having highly reviewed products have a few options for bringing their products to the forefront. They could pay to promote their product, artificially placing it among its top-rated peers. (These types of ads have been increasingly lucrative for Amazon, which sold $52.9 billion in ads in the second quarter of 2018 alone.) A cheaper — and perhaps more appealing — option is to fake it till they make it.
The shadowy world of fake reviews
The New York Times found plenty of examples of people offering fake reviews for sale on websites like Fiverr and Craigslist. “For $5, I will submit two great reviews for your business,” one Fiverr seller offered. “I will pay for positive feedback on TripAdvisor,” another person wrote on the advertising forum Digital Point. On Facebook, Amazon sellers offer money or free products in exchange for positive reviews, the Washington Post reported in April.
For a legitimate reviewer, there’s very little payoff to taking time out of your day to write about a product; it requires a certain degree of emotional investment, as Pavlou noted. But the act of writing a review isn’t difficult in and of itself — especially if you’re getting paid to do it. I could review a pair of headphones in just a few seconds — “Great sound quality, but they break easily” — without putting much thought into it.
This is why, according to Lappas, it’s so hard to spot fake reviews. “Think of a washing machine review. [One reviewer says] ‘This washing machine sucks,’ or ‘I wouldn’t recommend this product,’” he said. Whether those reviews are real or fake is “impossible to verify. There’s not enough text for you to look at it and know if it’s authentic or not.”
Most people can’t tell fake reviews from real ones, especially when the fraudulent reviews are written by real people. A 2011 study by researchers at Cornell University found that people could identify fake reviews just under 50 percent of the time — roughly the same level of accuracy as predicting a coin toss. And since it’s hard to determine which reviews are fake in the first place, it’s even harder to estimate what percentage of overall reviews were written by people who actually bought and used the products. Lappas estimates that 10 to 15 percent of online reviews are fake, though he said that the figure varies from platform to platform.
You may not see fake reviews when browsing for products, because algorithms tend to push “verified purchases” — those by people who actually bought the product — to the top, but fake reviews still affect the product’s overall rating, meaning they affect whether you see the product in the first place. Even as websites get better at filtering out fakes, unscrupulous reviewers find new ways to game the system. “It’s a cat-and-mouse thing,” Lappas said. “The defensive side gets better, and the bad guys get better as well.”
In October, for example, the skin care company Sunday Riley was caught encouraging its employees to positively review its products on Sephora’s website. Employees were reportedly sent a detailed email explaining how to avoid getting caught; tips included reviewing other brands’ products so as to not arouse suspicion and writing detailed reviews with lots of information.
“It helps to make yourself seem relatable — like you know how hard acne is and you’ve tried everything, and this one actually works,” the email read. “As reviews come in, read them too. If you notice someone saying things like I didn’t like ‘x’ about it, write a review that says the opposite.”
As Cheryl Wischhover wrote for Vox, Sunday Riley is by no means the only skin care company to flood the Sephora website with fake reviews. “Sephora puts the pressure on brands — they really ‘encourage’ reviews,” an anonymous beauty industry source told Wischhover at the time. “When you have a new launch, the sales will increase with a ton of good reviews. I do want to reiterate how common this practice for brands is. Sunday Riley just got caught. I’m not defending it, but it’s a vicious cycle.”
There’s also the fact that no matter how many positive reviews a product has, people will tend to remember the negative aspects of the few reviews they do read. In defense of its unethical reviewing practices, Sunday Riley claimed it was preempting negative reviews from competitors — behavior that both Pavlou and Lappas said is rampant in e-commerce, particularly in the beauty industry. “Most reviews are positive, and after a while, they get very redundant. They get very boring; they basically praise the same things over and over again,” Lappas said. “The customer loses trust in this. What they really care about is what customers don’t like about the product.”
The problem is not that reviews exist, but that people take reviews as gospel instead of as a list of helpful and not-so-helpful suggestions. The cult of online reviews has increasingly begun creeping into real life. Amazon is perhaps the most pervasive example. Amazon Books, the brick-and-mortar version of the e-commerce giant’s digital bookstore, displays customer reviews on its shelves. Amazon 4-Star, another IRL Amazon store, only carries products that have four or more stars on Amazon.
Writing product reviews is, in theory, a generous, selfless thing to do. In an ideal world, the reviewer gains absolutely nothing, aside from the satisfaction of helping a bunch of strangers on the internet make an informed decision. But the world we live in is far from ideal, so it makes perfect sense that the otherwise selfless system of product reviews has been co-opted by hucksters, frauds, and people who will lie to strangers in exchange for a $5 deposit into their PayPal account.
Want more stories from The Goods by Vox? Sign up for our newsletter here.
