From this summer, when you set up a new Android phone or tablet in Europe, you will be presented with an extra step: a choice screen to select your default search engine.
The screen is simple. Under a search icon and a short blurb are four options. One is always Google; the others vary depending on which country you’re in. Pick one of Google’s rivals, and its results – not Google’s – will appear in a home screen search widget and in the Chrome web browser. Its app will also be downloaded to the device.
The change is so subtle that many people may not notice it. But across the continent, it marks a fundamental shift. The choice screen appears in the wake of the European Commission fining Google €4.34 billion (£3.81bn) for breaching EU antitrust rules. On July 18, 2018, competition commissioner Margarethe Vestager hit Google with the fine – the largest of three she has issued against the company – for abusing the dominance of its Android operating system.
Being big isn’t illegal, but the European Commission says dominant companies have a "special responsibility" not to abuse their powerful market position by restricting competition. At the end of an investigation spanning more than three years, the Commission concluded that Google had acted illegally in using Android to ensure that traffic went to Google Search instead of its competitors. The Commission accused Google of three unfair tactics. Google, it said, had required manufacturers to pre-install the Google Search and Chrome apps if they wanted to license the Google Play app store; had paid mobile network operators and phone manufacturers to exclusively pre-install Google Search; and had prevented manufacturers from creating alternative versions of Android without its approval if they wanted to pre-install any Google apps.
Google’s actions, the Commission said, helped it to cement its dominance in the mobile search industry. In Europe, more than 95 per cent of all searches on mobile are made through Google. The market is mostly Android devices, but Google also pays billions to Apple to be the default search engine in the Safari browser on iOS. According to Vestager, its Android practices “denied rivals the chance to innovate and compete on the merits.” Google is appealing the decision.
The way many European antitrust fines work puts the onus of fixing a problem on the guilty party. Google has re-written the contracts it holds with phone makers and relaxed limits on how Android can be developed by others. On new phones and tablets shipped into the European Economic Area, it has introduced the search engine choice screen.
The choice screen gives Google’s rivals an unprecedented presence on Android devices. Smaller players have never had such an opportunity to directly reach users on the world’s biggest mobile operating system.
Chief among those set to benefit is DuckDuckGo, a US-based search engine founded in 2008 by CEO Gabriel Weinberg. DuckDuckGo will appear on all 31 choice screens across Europe when it is first implemented, a number matched only by Info.com. The company has been competing with Google for more than a decade, offering a search engine that emphasises privacy and doesn’t collect user data, but it is still tiny compared to Larry Page and Sergey Brin’s creation. “We're not trying to topple Google,” Weinberg, 40, says. “Our goal is for consumers who want to choose a private option, they should be able to do so easily.”
The idea for DuckDuckGo came partly from a stained glass class. In 2007, a couple of years after completing his masters in technology policy at MIT, and having already sold a social networking startup, Opobox, for $10 million, Weinberg decided to try his hand at a new hobby. The class teacher handed out a print-out of the best websites to learn more about creating stained glass. When Weinberg tried to search for information on Google, none of the recommended sites appeared in the top results.
The sheet of paper led to the creation of I’ve Got a Fang, a crowd-sourced site where people could contribute authoritative URLs on particular subjects. Weinberg, who lives in Philadelphia, believed the knowledge from people’s heads could be better than Google’s algorithms.
I’ve Got a Fang was a flop: few people added links to the site and it failed to get any traction. But the idea of human curation stuck, and Weinberg combined this notion with the remnants of another failed side gig – one that had already run into trouble with Google. Tldscan was a series of websites that crawled structured information and published it to the web – think sports statistics available in one place. In a now-archived 2010 blog post, “My history of (mostly failed) side projects and startups”, Weinberg wrote that he was getting 50,000 visitors and $500 revenue per day from Tldscan. “Then Google blacklisted all of my sites,” he wrote.
The combination of I’ve Got a Fang’s human expertise and Tldscan’s efforts to surface useful information became key pillars of DuckDuckGo, which launched a year later in September 2008. “My original spark was: could there be a better user experience in search?” says a bespectacled, clean-shaven Weinberg, speaking via a Zoom call in March. (While many companies had to radically adapt their way of working as the Covid-19 outbreak spread across the world, little changed for DuckDuckGo; its 89 staff work remotely all the time, and there is no main office.)
Initially, Weinberg started building the technology needed to crawl, index and rank the web himself, but he stopped when Yahoo! introduced a service called Build Your Own Search Service (BOSS). Using this, he focused on refining Yahoo!’s results and adding extra services on top. Today, DuckDuckGo combines data from more than 400 sources into its search results: Microsoft’s Bing is predominantly used to surface relevant pages, but Wikipedia, Apple Maps, TripAdvisor and DuckDuckGo’s own web crawler also contribute. There are more than 1,000 different sources for the “Instant Answer” snippets DuckDuckGo shows alongside its results.
It wasn’t until 2010 that the search company set out its unique selling point: it wasn’t going to track users. The company committed to throwing away IP address information and not storing details of what people searched for. “At the beginning, privacy wasn't the main draw,” Weinberg says. “It wasn't as known as an issue.”
Initial growth was slow; in April 2010 the site was getting around 30,000 to 40,000 search queries a day, barely appearing as a blip on the search radar. Weinberg didn’t hire his first full-time employee until 2011, and up until then he was primarily focused on being a stay-at-home dad for his first child, born the year after DuckDuckGo launched. During some early investor meetings, he says, he was rocking his second child out of his webcam’s field of vision. The company raised $3 million in a Series A round in 2011, led by Union Square Ventures.
DuckDuckGo’s real breakthrough came thanks to Edward Snowden. The former NSA contractor’s explosive government surveillance revelations in 2013 prominently included Google among the data sources used by British and US spy agencies. People were newly interested in seeking out privacy-first alternatives, and DuckDuckGo’s daily search queries swelled from 1.5 million a day to 4.1 million a day in the space of a year.
At the time, many people switching from Google wouldn’t have been happy with the experience their new search engine offered, and it’s likely that plenty quickly switched back. Weinberg says that it wasn’t until 2014 that DuckDuckGo could really compete with its Silicon Valley rival. It was a crucial year for the company: it hit its tenth employee; incorporated news, video and image verticals in a big site redesign; and was included as an alternative search engine option in Apple’s Safari browser on iOS. “Apple is effectively a mainstream endorsement,” Weinberg says.
It was in 2014 that DuckDuckGo also made its first profit; it’s been profitable ever since, and the company forecasts it will pass the $100 million revenue barrier for the first time in 2020 (thought this may be affected by the Covid-19 pandemic).
It makes its money in the same way as Google: through advertisements. Each time a user clicks on an ad that is shown alongside search results, the company takes home a small amount of money. But where the two differ is that DuckDuckGo uses contextual advertising over the data-heavy behavioural advertising that helps Google earn billions each quarter.
Contextual adverts are shown to users based on the searches they make: if you DuckDuckGo “Mercedes”, you’ll be served adverts for cars on that page. Google “Mercedes”, however, and adverts for cars will follow you off the page and around the web, because Google adds that data to the other information it knows about you. Behavioural advertising is much more lucrative, as each time an ad is clicked on there’s a higher likelihood the person will buy. Google’s expertise in knowing what users will be most likely to click on is rivalled only by that of Facebook.
People’s increased awareness of privacy issues has continued to fuel DuckDuckGo’s growth. As with Snowden’s revelations, the 2018 Cambridge Analytica scandal propelled Weinberg and DuckDuckGo into the mainstream. Around the same time, the company launched desktop browser extensions that stop web tracking, as well as its own mobile web browser for Android and iOS.
The cumulative effect was significant. On March 17, 2018, the day The Observer first reported on Cambridge Analytica, 17 million DuckDuckGo searches were made; exactly two years later, 57 million searches were completed. Weinberg says DuckDuckGo’s key users are people who both care about their privacy and are willing to act upon it. “The caring really went up after Snowden,” he says. “After Cambridge Analytica the acting percentage has really skyrocketed.”
Google’s new Android choice screen could give another boost to DuckDuckGo, finally propelling it into the mainstream as millions of people replace their old phones over the coming years. “It's really a levelling, if it's done right, for competition,” Weinberg says. But he, and other search engines, have issues with Google’s implementation.
Multiple European search startups say they are in favour of a choice screen. They argue that users should be allowed to pick which search tool they use – even if they end up choosing Google (in most cases they will). But Google’s execution of the choice screen has angered its rivals.
To appear, companies have to pay Google, via an auction that is held every three months. Bid enough, and you win a coveted slot as one of three options alongside Google. A similar auction approach was introduced as a fix to a 2017 EU antitrust case around Google Shopping. In search, companies say they don't think an auction is a fair way of encouraging competition or innovation.
Google argues that an auction is a “fair and objective” way of smaller companies being able to compete, and that it allows them to decide “what value they place on appearing in the choice screen”. Every time an Android user picks a search engine to use – even if they switch back to Google minutes later – that search engine is charged. Google issues the companies an invoice each month, and the money goes back into the development of Android.
Ahead of the first round of bidding towards the end of 2019, Google set a minimum bid. Multiple sources, who are not able to speak publicly, say it set prices of above $20 per user for some European countries. This amount would be unrealistic for small companies, the sources say. “I actually think they saw this as a price we would easily be able to pay,” one person familiar with the bidding says, adding that it was “ridiculous”.
Ultimately, the minimum bid was scrapped in favour of a system where all the winners pay the price set by the first losing bidder. Google says it changed the format after receiving feedback and to make it easier for rivals to participate. One source says companies will now be paying the “lowest number to zero that you can do besides being zero.” Another company says the only reason it bid in the process was because it was cheaper than advertising on Facebook. “We will get our name out more than I think we will directly get users from the choice screen,” the company CEO says.
DuckDuckGo is one of the auction’s big winners, appearing as an option in all 31 countries. But it isn’t happy about it. “You're incentivised to bid your profit,” Weinberg says. He predicts the auction prices will keep rising until it’s harder for search engines to make any money. “Google is taking away all the profit of these search engines and profiting from it themselves,” he says. “They're effectively running other people's search and taking other people's profit.”
“It plays the competitors against each other,” says Christian Kroll, the CEO of Ecosia, a German search engine that plants trees when searches are made on its platform. “So instead of collaboration, we have now kind of a toxic environment between smaller search engines as well.” Ecosia decided to boycott the auction. Kroll says in the long term he doesn’t believe the company can win, because 80 per cent of its profits are spent on tree planting. “I think this was very well thought out by Google,” he says.
DuckDuckGo has loudly criticised Google’s efforts. Weinberg and DuckDuckGo general counsel Megan Gray, who previously worked at the US Federal Trade Commission in consumer protection, argue that Google’s process is flawed. They say there should be more than four options shown to users, and that people should be given more information about the choice they’re making. If there was one fix? “Getting rid of the damn auction,” Gray says. “It should be a genuine choice – it shouldn't be pay to play. That would be the best outcome here.”
Paul Gennai, a product management director at Google, says that Android has “created more choice for everyone, not less” and that the operating system's openness has meant “developers of apps that compete against Google have precisely the same ability to reach users”.
“From day one, Google developed Android to be open: both to the manufacturers who build devices, as well as to the people who buy them – we think both should have the flexibility to set them up just as they want,” Gennai says. “This has taken billions of dollars of investment over more than ten years.” He adds that Google also invests in its Play app store, which includes safety features such as scanning apps for malware. “With the changes required in Europe, we implemented an auction to ensure that it would continue to be viable for Google to continue its investments in the open Android ecosystem,” he says.
A spokesperson for the European Commission says it is closely monitoring how the choice screen is being introduced and the mechanisms behind it. "We have been discussing the choice screen mechanism with Google, following relevant feedback from the market, in particular in relation to the presentation and mechanics of the choice screen and to the selection mechanism of rival search providers," the spokesperson says. (Since this story went to press, the results of the second Android choice screen auction have been published. Fewer companies are included as winners and Microsoft's Bing has no presence. DuckDuckGo says prices have increased, too).
Aside from controversies around the auction, there are questions over how much difference the choice screen will make. A 2010 EU antitrust investigation saw Microsoft launch a choice screen for web browsers – the Commission had said that Microsoft used the dominance of the Windows operating system to direct people to its Internet Explorer browser, which was included as the default on Windows. Its choice screen ran for four years, but at the end of this time smaller browsers had not gained any significant market share. (The biggest market shift was the rise of Google's Chrome browser, which had only recently launched and was rising in popularity anyway.)
And in search, Google nearly always wins. One major exception – aside from in China, where Google search is banned – is in Russia. There, Yandex is king. Launched in 1997, a year before Google, Yandex acts as the country’s homegrown Uber, Spotify, Amazon and favoured search engine all rolled into one. At the start of 2015, Russians searching the web on their desktop machines were overwhelmingly likely to be using the company – 63 per cent of all searches were made using Yandex. But on mobile it was a different story. In an Android-dominant country, the tables were turned, with 62 per cent of all mobile searches being made via Google.
Staff at Yandex were aghast when they discovered Google had struck deals with phone makers that prohibited them from accepting payments to pre-install other search engines if they wanted to use the Android operating system.
As in the EU, regulators in Russia were scrutinising the power Android gives Google search. In March 2017, the Federal Antimonopoly Service (FAS) fined Google 438,067,400 roubles (£4.4 million) and then reached a settlement with the company that said Android would offer a choice screen for search engines, in this case on both new and existing devices (Google says technical changes mean the EU choice screen can only apply to new phones).
Smartphone users in Russia started to see choice screens from August 2017. That month, search on Android was 60 per cent Google. The gap closed and a year later it was split 50/50 with Yandex. At the start of March 2020, Yandex had 56 per cent of all Android searches, with Google’s share dropping to 42 per cent.
But while Yandex was also one of the bigger winners in the EU Android auction, appearing on device screens in 16 countries, Preston Carey, a senior vice president at Yandex, says he doesn’t expect to see any large market shifts. “Putting a choice screen up in the UK or Germany is not going to have the same result as putting one up in Russia,” he says. He argues that no other search company has the brand recognition that Yandex does in Russia in any other country – meaning people will be less likely to switch away from Google.
In its own tests, DuckDuckGo created a choice screen with 18 different options and placed Google at the end of the list. It found people still scrolled all the way to the bottom to find the one they were most familiar with.
While Android’s new choice screen is unlikely to end Google’s dominance in search, even a one per cent shift in market share over a period of several years would be colossal growth for firms such as DuckDuckGo. Weinberg predicts his firm could one day have a market share of ten per cent (it currently sits at 0.32 per cent of global market share). This would still pale in comparison to Google but be a big boost to the 1.5 billion monthly DuckDuckGos already taking place.
And to DuckDuckGo, the battle for search represents just a small part of a bigger struggle over how advertising on the web works, and what this means for personal data and privacy.
The vast ad networks of Facebook and Google are fed with personal data collected across the web. Trackers from both companies gather information about online activity from almost all of the web’s most popular sites. By collecting billions of data points, the companies are able to build revenues in excess of $20 billion every three months. The vast majority of this money comes from advertising – in particular behavioural advertising.
Trackers and cookies are present across almost all websites; they monitor and track people’s activities online and log them under unique identifiers connected to individuals (Wired.co.uk uses trackers to collect and share reader data as part of business practices). But at the start of April this year, the data protection regulator for Ireland, which oversees all the big US tech companies headquartered in the country, released a report saying many websites fail to give “basic information” to people that visit them and that “most ordinary users will not be aware of the extent to which they may be tracked”.
This tracking allows companies to collect data on people’s interests and activities, which they can feed into advertising models and use to serve people behavioural adverts linked to their interests; Facebook’s Pixel, for example, is on more than eight million websites. “Strategically, Google and Facebook have all the user data,” Weinberg says. DuckDuckGo’s web browser, like its search engine, does not store personal data. In addition, it blocks the invisible trackers hidden on web pages. The company wants to see less tracking elsewhere, too.
In 2009, security researchers proposed a Do Not Track setting for browsers. In theory this is a checkbox, or an option in a browser’s settings menu, that allows a user to opt out of being tracked as they move around the web. Firefox, Internet Explorer, Opera and Chrome all use it, but it's largely ineffective. (Apple's Safari stopped using it in 2019). Websites across the web don’t have to honour the setting; Medium and Pinterest are two of the most prominent that do. Twitter originally supported it but doesn’t anymore, and efforts from web standards bodies have failed to make it more effective. Despite having the feature in Chrome, Google’s support pages say the company’s own websites “don't change their behavior” when Do Not Track is turned on.
In 2018, consulting firm Forrester found that around a quarter of Americans had the largely-ineffective setting switched on. A year later, lawyers for DuckDuckGo proposed “The Do-Not-Track Act of 2019”, a draft piece of legislation designed to make the Do Not Track setting be backed by law: websites would have to respect a user’s wishes to not have their behaviour online recorded.
The idea has not been put into practice anywhere in the US but a similar concept is marginally closer to reality in Europe. For the last three years, European legislators have been debating the ePrivacy Regulation, a complimentary law to GDPR which could include Do Not Track provisions. The regulation has been subject to intense lobbying and is currently in a state of stagnation. But it’s the closest anywhere has got to limiting how people are tracked online.
Weinberg believes its enactment would open up the online advertising market and thus potentially reduce the dominance of Facebook and Google in the space. If a quarter of the web was suddenly using a setting that meant it was harder to serve them adverts targeted at their interests, traditional contextual advertising would rise again. This would be good for privacy, but it could also help open up competition, Weinberg says; if Google and Facebook didn’t have such a data advantage, perhaps smaller companies could compete on a more even footing. “If everyone competes on contextual advertising, now all of a sudden they actually have to compete with a lot of other players,” he says. Google has already been experimenting further with contextual advertising, which one of its product managers wrote in a blog post was to help preserve user privacy.
Competition regulators are also shifting their focus to how data is used by big tech companies. On both sides of the Atlantic, examinations are underway into how Alphabet, Amazon, Apple, Facebook and Microsoft have acquired smaller companies. The EU is also looking more broadly at how Google and Facebook collect data and the implications for competition.
Ultimately, Weinberg wants people to understand that they don’t need to give away their personal information to make the most of technology. “We really want to combat people's learned helplessness in privacy,” he says. “Everyone's concerned, but a large percentage of people just think there's nothing you can really do about it. We want to prove that you can do something.”
Updated: This article first appeared in print. Prior to online publication Megan Gray's role at the FTC has been updated and Gabriel Weinberg was rocking his second child while talking to investors
Matt Burgess is WIRED's deputy digital editor. He tweets from @mattburgess1
This article was originally published by WIRED UK
