It’s a truism that we live in a “digital age”. It would be more accurate to say that we live in an algorithmically curated era – that is, a period when many of our choices and perceptions are shaped by machine-learning algorithms that nudge us in directions favoured by those who employ the programmers who write the necessary code.
A good way of describing them would be as recommender engines. They monitor your digital trail and note what interests you – as evidenced by what you’ve browsed or purchased online. Amazon, for example, regularly offers me suggestions for items that are “based on your browsing history”. It also shows me a list of what people who purchased the item I’m considering also bought. YouTube’s engine notes what kinds of videos I have watched – and logs how much of each I have watched before clicking onwards – and then presents on the right-hand side of the screen an endlessly-scrolling list of videos that might interest me based on what I’ve just watched.
In the early days of the web, few, if any, of these engines existed. But from 2001 onwards they became increasingly common and are now almost ubiquitous. Several factors drove this expansion. One was the need to help users cope with the information overload that came with the web: recommender engines could sift through the torrent and create a personalised distillation just for you. But the prime driving force was the business model we now call surveillance capitalism – logging our online behaviour in order to make increasingly refined predictions about our preferences and likely needs that could be sold to advertisers anxious to sell us stuff.
When social media started, each user’s news feed consisted of a simple chronological listing of what their friends had posted. But in September 2011 on Facebook, all that changed: from then on users’ news feeds were “curated” by a machine-learning algorithm. Mark Tonkelowitz, an engineering manager at Facebook at the time, described the curated news feed thus: “When you pick up a newspaper after not reading it for a week, the front page quickly clues you into the most interesting stories. In the past, News Feed hasn’t worked like that. Updates slide down in chronological order so it’s tough to zero in on what matters most. Now, News Feed will act more like your own personal newspaper. You won’t have to worry about missing important stuff. All your news will be in a single stream with the most interesting stories featured at the top.”
It turned out that some of those “interesting stories” were of great commercial interest to Facebook because they encouraged users to engage with the content – and therefore prioritised it. Since 2016, we’ve become increasingly aware of how this algorithmic curation can be used to induce us to buy not just goods and services, but ideas, mis- and disinformation, conspiracy theories and hoaxes, as well.
For years, I fondly imagined that curation of ideas was the business only of social media. But an article last year by Renée DiResta, a leading expert on online misinformation, suggested that the phenomenon goes beyond Facebook et al. Scrolling through a simple keyword search for “vaccine” in Amazon’s top-level books section, she found “anti-vax literature prominently marked as ‘#1 Best Seller’ in categories ranging from Emergency Pediatrics to History of Medicine to Chemistry. The first pro-vaccine book appears 12th in the list. Bluntly named Vaccines Did Not Cause Rachel’s Autism, it’s the only pro-vaccine book on the first page of search results.”
Over in Amazon’s oncology category, DiResta found a book with a bestseller label touting juice as an alternative to chemotherapy. For the term “cancer” overall, she noted that The Truth About Cancer, “a hodgepodge of claims about, among other things, government conspiracies”, had 1,684 reviews (96% of them five-star ones) and was given front-page placement.
Just out of interest, this week I tried a search in the books section on Amazon.co.uk for “cancer cure”. Of the first 11 available titles that came up, only one looked like a conventional scientific treatment of the topic. The others focused on herbs, oils and “natural cures they don’t want you to know about”. This is not because Amazon has a grudge against scientific medicine, but because there is something about unconventional books in this area that its machine-learning algorithm is detecting – perhaps from reviews posted by evangelists for non-scientific approaches. (DiResta thought that this may indeed be the explanation: Amazon did not confirm this.) But it’s conceivable that in really controversial – and currently topical – areas such as vaccination, coordinated user reviews by anti-vaxxers might successfully game the algorithm. And in the past Amazon has been accused of being “a giant purveyor of medical quackery”.
What it really means, I guess, is that, in the online world, information warfare is now ubiquitous. And since books are really just containers for information and ideas, it was predictable that marketplaces such as Amazon would become targets for manipulation. Truth is always the first casualty in war.
What I’ve been reading
The double bind of fake news
Ted Gioia’s marvellous LA Review of Books essay on great British scholar Gregory Bateson.
Down for the count
Sobering essay on Aeon by Harvard historian Arunabh Ghosh about the Chinese Communist party’s decision to reject statistical sampling in favour of exhaustive enumeration.
Next generation technology
Roboticist Rodney Brooks’s speculative blog post on which pillars of current scientific wisdom will crumble in the lifetime of his grandchildren.