--
Every weekday beginning at around nine, 30 contracted white-collar laborers arrive at a generic office park in the outskirts of Knoxville, Tennessee. They are in their twenties and thirties, more or less evenly split in gender. They head to a three-story building that hosts the outposts of multiple companies and head to a space modeled on the open floor plan of the main Facebook offices in Menlo Park, California, with walls dotted with some of the same posters that hang at One Hacker Way.
They sit down at desktop computers and get to work, doing something that hundreds of millions of people do every day for free — going through their personal Facebook News Feed and deciding which stories they like. They are the advance guard of a project that now extends to around six hundred people around the country, performing the same tasks for four hours a day. Eventually, it may reach — in a diluted form — every Facebook user.
Essentially, Facebook has determined that algorithms alone are not enough to determine the makeup of one’s News Feed. The mix is so important — not just to individuals but to collateral players such as the news business, the apps industry, and the Internet meme machine — that the ultimate tech company must acquaint itself with the Antediluvian art of asking people directly what they want.
It’s part of a continuing effort by Facebook to make the News Feed central to our existence. “The dream is to get to this world where people feel that Facebook is an instrumental, useful, important part of their lives,” says the company’s chief product officer, Chris Cox. “That’s the golden thing.”
Facebook has made considerable progress in fulfilling this dream. When it first appeared in 2006, the public response focused on privacy concerns — some people were shocked that news of activities they had shared was suddenly pushed out to all their “friends” — and 10 percent of the Facebook population joined a group protesting News Feed. But after some apologies and tweaks it quickly became Facebook’s most popular feature by far. That was by design: according to Cox the idea was to make the stream of stories into Facebook’s home page, personalized for each user.
In News Feed’s early days, Facebook expended surprisingly little effort on ranking the stories: “At the beginning we simply used our own intuitions and our own experiences—that’s all we had,” says Cox. It wasn’t until a couple of years ago that Facebook began making News Feed rankings a top priority, using feedback loops, machine learning and analysis of how close each user was to the author of each story. By then, Facebook had clarified its thinking on the News Feed to make it, as Cox says, “a really amazing starting point to launching a digital universe.”
“The goal is to build the perfect personalized newspaper for 1.1 billion people and counting,” CEO Mark Zuckerberg told me in 2013. (The most recent number is 1.39 billion.) “That involves all these different kinds of content, and it involves figuring out what types of stuff people care about, which people they care about and the topics they care about.”
If people come to perceive the hours they spend on the News Feed as less than rewarding, they will inevitably spend less time on it, if not abandon it altogether. Thus Facebook is fully committed to continual improvements. Mimicking the mechanics by which Google has evolved its search engine (there are a lot of ex-Googlers at Facebook), the News Feed team proceeds on two fronts: modest incremental improvements rolled out at weekly launch meetings (things like the rate at which an unread story gets “bumped” up in the timeline) and more significant algorithmic changes. For instance, last August an update targeted clickbait, and just a couple of weeks ago another update integrated tools to prevent hoaxes — those amazing stories that become much less interesting when revealed to be totally made up.
The problem for Facebook is that user metrics have become a feedback loop for useless diversions. Though Facebook has gotten very good at the gold standard of stories — delivering important news about marriage, childbirth and exotic vacations of close friends — people’s news feeds (well, mine) have been overpopulated with listicles (50 Most Bizarre Prom Pictures!), animal videos (I Put A Go-Pro On My Dog And Left the House!), and political red-or-blue meat (Sarah Palin, Sarah Palin, Sarah Palin).
You could call this the Dozen Doughnuts problem. Many people conscious of their weight know it’s not a good idea to eat a doughnut every day, and if given a choice would not prefer that someone come into the workplace every morning with twelve Krispy Kremes. But if a misguidedly generous worker did just that, the temptation to pluck one of those jelly-filled delights might overrule discretion. It’s not that you want the doughnut—you aren’t clamoring for one, and you won’t miss that sugar bomb if it’s not in front of your face. But once that delicacy is in front of you…oh, what the hell!
For lots of us, the Facebook News Feed is a never ending delivery of info-doughnuts — empty calories of celebrity misdeeds, snuggling animals of different species, and quizzes that guess where you’re from (what, you don’t know? But we take the tests!). And when we do click on them, we send a strong signal to Facebook’s algorithms that we want to see those things. We clicked, didn’t we? And, as Facebook’s engineers and managers constantly explain, the company is nonjudgmental about what’s in anyone’s News Feed — as long as it makes the user happy.
“We really try to not express any editorial judgment,” says Adam Mosseri, the News Feed product director. “We might think that Ferguson is more important than the Ice Bucket Challenge but we don’t think we should be forcing people to eat their vegetables even though we may or may not think vegetables are healthy.”
But what if the feedback loop didn’t account for unexpressed desires for a News Feed with more kale smoothies? Despite its professed neutrality, one suspects that Facebook’s leaders privately regard with dismay the company’s role in sugar-bombing the news industry, as editors and writers optimize its story selection and presentation to be irresistible when they pop up in News Feeds.
Grappling with such concerns early last year (as well as its continuing mania for improving the product), Facebook concluded that the algorithmic approach alone might be too limited for a truly satisfying News Feed experience. Sometimes, you just have to ask people.
“It comes from the intuition that you can only get so far by looking at online behaviors,” says Cox. “It’s expensive, and it takes time. But what you really want is to sit down with 1.2 billion people, every single one, and ask them to go through and point at ‘I really loved that one.” Why did you really love that one? ‘Well I really liked that one because it’s from a person I went to high school with and I use Facebook to stay in touch with people from high school.’ Why did you hate this one? “I really hated this one because I really hate memes.’”
“If you just watch people eat doughnuts, you’re like, ‘People love doughnuts, let’s bring them more doughnuts,” says Greg Marra, a News Feed product manager. “But if you talk to people they’re like, ‘No actually what I want is to eat fewer doughnuts and maybe eat a kale smoothie….’ Then we can give them some kale smoothies, too.”
That level of interrogation won’t work for a billion people. But it could certainly be done with 30 people, who would become a pilot project for a broader interlocutory examination of the Facebook population.
And that’s why Facebook hired a testing firm to set up its professional raters experiment in Knoxville.
Here’s how it works: the raters first click on a button in a special version of Facebook, and are instantly presented with what Facebook considers the 30 top News Feed stories for their individual personae on the service. One difference from the norm is that Facebook does not rank those stories in order—they are presented randomly. Also, unlike the usual Facebook user, who will look at only a few stories at a time (though eventually going through around 100 stories in the course of the day), these paid raters go through each story one by one.
First they interact with it as they would if they were using Facebook at home — either ignoring the story or engaging with it, by commenting, sharing, or following links. Then, for each story, the raters answer eight questions about how they felt about the story: stuff like how much they care about the person in the story, how welcome the story was in the News Feed, how entertaining they found the story, and how much the story connected them to friends and family. Finally, they must write approximately a paragraph explaining their overall feelings regarding the story. In addition, sometimes Facebook personnel from the Menlo Park mothership visit and quiz raters individually on their reactions to stories. “It’s a pretty intense process,” says Max Eulenstein, a News Feed product manager who has been shuttling back and forth to Knoxville since the project started on August 18, 2014.
More recently, Facebook has expanded the project to 600 people around the country, working four hours a day from home. Those numbers will soon expand to the thousands. Finally, Facebook envisions some sort of direct questioning to its entire population.
“We can ask people what topics they’re interested in, and promote things about sports if you like sports, demote news if you don’t like news,” says Mosseri. “Some of those insights will be useful in evaluating very specific ranking changes, and some will be useful in terms of product changes.”
But even in these early stages, the News Feed team says that the project is yielding some valuable findings. According to Mosseri, Facebook wants News Feed to do three things: “One is connecting you to your friends and family—that’s the fundamental value that Facebook is based on,” he says. “The second is to inform about things that you might be interested in, whether it’s news or sports scores or how to wash your jeans. And the third thing is to entertain you, whether it’s make you laugh or show you videos or trailers.” By pigeonholing content in those buckets, he says, Facebook is able to judge which kinds of stories its raters welcomed, and why.
Most of the nascent results have tended to confirm some of Facebook’s current hypotheses about News Feed behavior. If it wasn’t already clear enough, the raters provided further evidence that nothing tops vital news from close friends. “The stories we do best on are tagged and photo stories,” says Eulenstein. “Our rankings get better and better the more a story is about a person and the less it’s about an article.”
In fact, the impact of close friends even extended to the way people reacted to stories posted by BFFs and relatives that don’t necessarily involve personal news. Though this is intuitive, the News Feed team feels that they have been underestimating this phenomenon, and may have to adjust the algorithms accordingly. One observation from the Knoxville testers has been the ambiguous meaning of a “like.” Clicking that button to presumably express approval on a story has different import depending on your relationship to the author.
If the story is posted by a close friend or relative, the “like” impulse doesn’t really reflect your approval of what appears, but fortifies your connection with the author. It’s almost a dis not to like it. But the “likes” you apportion to stories by acquaintances actually reflect your interest in the link or picture. Thus, when trying to divine what stories should be highly ranked in your News Feed, Facebook learned that it might want to discount the subject matter of stories by close friends. (But it should still give you stories posted by your intimate cohort, because people want to acknowledge what friends send them.)
Also, the testers’ evaluations showed that Facebook still has a long way to go before reaching its stated goal of making sponsored stories (i.e., ads) as welcome and useful as other posts in the News Feed. “It’s as expected,” says Eulenstein. “In general, commercial content is less desirable than other forms of content.” (Just wondering—if “less desirable” ads are already bringing in the lion’s share of Facebook’s $12.5 billion annual revenue, how much will they reap if people actually crave News Feed sponsored stories?)
One of the big questions, of course, is whether this study would indicate that people are yearning for more meaningful stories — the kale smoothies. So far, the answer is no. “If anything it’s the inverse,” says Mosseri. “When we asked what are the best stories, ones people said they really want to see, the highest percentage of impact type is a strong emotional reaction. People really want to see stuff that drives a laugh or makes them feel happy, not necessarily information that’s super valuable.” On a story-by-story basis this instant-gratification impulse may even overwhelm a desire to hear news about people in one’s social graph. “Most people say I want to see more [stories] from my friends,” says Eulenstein. “But at the individual story level we’re getting signals that suggest people want to see more content that’s public.”
So does this negate the Dozen Doughnuts theory? Not quite. Mosseri says that the current results reflect the fact that so far testers have only been asked to rate individual stories — the next step will be to globally assess sets of posts. “If you ask people about each story individually they’re going to naturally rate emotional reactions really highly, which is what we’re seeing,” he says. “Whereas once we start to ask about sets of stories, our hypothesis is that people will start to ask for more friend content and more informative content.”
We can only hope. If that desire exists, Facebook is determined to find it. According to the company, The Knoxville project will continue for the foreseeable future. “Months, maybe years,” says Mosseri.
Story by story, Facebook is learning more about what we want in our News Feed. Eventually, we’ll have no one to blame for those doughnuts but ourselves.
