When a political ad goes viral on Facebook, conventional wisdom holds that it was a success. After all, the Golden Rule of advertising in the digital age is simple: Engagement is good. It’s good for Facebook too. The more time users spend watching, commenting, clicking, and sharing on its platform, the more money the company makes. Little wonder, then, that Facebook allows advertisers to test which ads get the most engagement with a single click.
That model works just fine when you’re trying to get people to donate or volunteer for a cause. It’s easy enough to see whether all those impressions translate into more donations and sign-ups. These types of ads are known as direct-response ads, because they urge the target to take some action.
The calculus changes when it comes to persuasion ads, which aim to build support for a certain candidate or issue among people who haven't made up their minds yet. The key to persuasion is not simply engaging the biggest audience but successfully moving the right one. Still, lots of political advertisers, including campaigns and advocacy groups, use engagement as a proxy for persuasion. That's particularly true when they don't have the means for more in-depth poll testing. Sometimes we in the press also conflate the two, believing that virality and persuasiveness are somehow synonymous.
But is that true? Or, in the age of echo chambers and filter bubbles, is high engagement really just a sign that you’re preaching to the converted?
That's the question a group of progressive digital strategists working out of Harmony Labs, a nonprofit dedicated to studying media influence, set out to answer in a series of recent tests. The study wasn't peer-reviewed, and it was conducted largely for Harmony Labs' internal purposes. But its findings may still raise alarm bells for political groups that have gone all in on Facebook after witnessing Donald Trump’s success there during the 2016 presidential election. Not only did the group find zero correlation between engagement and persuasion; in some cases, the most engaging videos persuaded people in the wrong direction.
“The thing the platforms are driving you to is increasing their revenue, but it doesn’t necessarily persuade the people that need to be persuaded,” says Nathaniel Lubin, former White House digital director under President Barack Obama. Lubin designed the experiments along with Peter Koechley, cofounder of UpWorthy, whose founding mission was to make viral content promoting political and social issues.
Through Harmony Labs, Lubin and Koechley recently launched a project geared toward sourcing and creating left-leaning advocacy content that’s both engaging and persuasive. They call it the Meme Factory. To test this theory about persuasion, the Meme Factory team created a series of 18 videos covering a range of key progressive issues, from opposition to the GOP tax bill to the repeal of DACA, the Obama-era program that protected certain undocumented immigrants who came to the United States as children from deportation. The videos were conceived and recorded just as they might be within a cash-strapped political organization, with minimal production requirements and quick turnarounds.
For each political issue, they created several videos, some preachy and heartrending, others funny and filled with stats. Then they set the videos loose on Facebook, targeting them as ads to key audiences. The DACA videos, for instance, were aimed at college-educated women in the suburbs of dozens of US cities. Lubin and Koechley logged the engagement on each using several different metrics, including how long people spent watching the video and how many times they liked, clicked, or shared it.
In lots of campaigns, the work might stop there, and the bulk of the advertising budget might be thrown at whichever video people watched most. But the Meme Factory team took it a step further, contracting a startup called Swayable, which measures media content's persuasiveness through randomized controlled trials. Its CEO, James Slezak, is both a trained physicist and the former executive director of digital strategy at The New York Times.
For the Meme Factory project, Swayable sourced study participants from MTurk, dividing them into groups that mirrored the audiences Lubin and Koechley targeted on Facebook. For each video, there was a control group and a test group. The control groups were shown what Slezak calls “placebo content,” like a public service announcement about texting and driving. The test group watched one of the Meme Factory videos. Then both groups were given a survey asking whether they agreed with the video and whether they’d take action on the issue. If the test group agreed more than the control group, then Swayable deemed them, well, swayed. All together, the experiment surveyed 10,000 people, amassing 100,000 data points about them in total, including basic information about demographics and partisan affiliation.
One of the most engaging DACA videos was called “Heartache,” a tearjerker of an ad that featured crying children being separated from their undocumented parents. It blamed “Donald Trump’s America” for their sorrow and asked viewers to share the video if they think America is better than this.
Far less engaging was “What Would You Do?”---a split-screen video of a woman watching another video about the DACA debate. It’s loaded with snark, pop culture references, and emojis.
In the end, the videos' success on Facebook, or lack thereof, turned out to have little impact on their ability to persuade. Swayable found that after watching "Heartache," the people who identified as most conservative were less supportive of DACA than the control group. The "What Would You Do?" video, meanwhile, elicited more support for DACA among even the most conservative participants, compared to the control group.
But those are just two examples. Swayable plotted the persuasion and engagement scores of dozens of combinations of videos and audiences, and found no correlation whatsoever. “There’s nothing but just random static,” Slezak says.
.jpg)
Of course, this may not come as news to more sophisticated campaigns. During the 2016 election, Trump’s digital team spent the bulk of its Facebook advertising money on direct-response ads, geared toward getting supporters to donate or turn out to vote, rather than persuading undecided voters to support Trump in the first place. They conducted “A/B testing on steroids,” as former Republican National Committee director of advertising Gary Coby told WIRED shortly after the election, running up to 175,000 variants of an ad in a single day to see what worked. The approach was so successful, Facebook has adopted the Trump playbook for its own advertisements, according to a report by BuzzFeed News.
But Coby says focusing too much on engagement is a mistake. "Engagement alone should not be your key performance indicator for a persuasion or direct response buy. It's very shortsighted if people are thinking that alone can guide their optimization," he told WIRED this week. "During Trump 2016, I had controlled studies running to measure the performance of our persuasion buys on Facebook."
Facebook says it hasn't studied the link, or lack thereof, between persuasion and engagement. For advertisers that are already spending a lot of money on Facebook, the company conducts what it calls brand-lift studies at no additional cost. But small campaigns aren't usually eligible. For those that aren't, Facebook spokesperson Andy Stone says, "We encourage advertisers to test and analyze based on their prescribed goals." But Facebook's self-service advertising tools---the ones that allow anyone to place an ad in minutes---don't give users a way to analyze persuasion the way they can analyze engagement. Stone says Facebook is considering making brand-lift surveys available to more of these users in the future. For now, though, smaller advertisers need to rely on costly focus groups, which don't offer the real-time insights that make digital advertising appealing.
So, relying on engagement becomes an easy trap. Lubin worries that is driving advocacy groups to allocate their limited resources inefficiently as they work to recruit new supporters. What’s more, Slezak argues that investing in content that’s only meant to engage people runs the risk of making our already hyper-polarized politics even worse. While researchers continue to debate the impact of social media filter bubbles, it's undeniable that platforms like Facebook play an increasingly prominent role in our elections.
“If we pick content to promote based on how well it engages, what we’re really measuring is how much it echoes around our own echo chambers,” Slezak says. “If we want to win over people who are not like us, we have to find ways to listen to them and hear whether our content or messages are resonating.”
- Pledging openness, companies still rush to patent AI tech
- Naked Labs' 3-D body scanner shows you the naked truth
- Why Westerners fear robots and the Japanese do not
- A deadly treasure hunt spawns an online mystery
- Could the US electric grid go the way of the landline?
- Looking for more? Sign up for our daily newsletter and never miss our latest and greatest stories
