To live in the Information Age sounds fun and, for the most part, it is. In this age, data is becoming increasingly easier to collect. But as we get more and more of it, to organise this data into information we can get meaning and value from becomes a challenge in itself. Here, I’ll walk you through the failure that was eBay’s early-day search, and what we can learn from it.

We’re entering a new world in which data may be more important than software. — Tim O’Reilly

By the early 2000s, eBay was growing rapidly with more than 12 million registered users and over 4.5 million items on sale on any given day. Online shopping was growing as people joined the internet revolution. It was at that time that the company entered into fierce competition with the other major disruptor in the market — Amazon.

The two companies approached the situation in completely different ways. Whilst Amazon was focused on improving the buyer experience, eBay was primarily involved in creating value for sellers. That crucial difference in strategy ultimately made Amazon the global leader that it is today. So, what went wrong for eBay?

Search

The problem

At first glance, both eBay and Amazon offered a similar search experience: the buyer would go on the platform’s website, enter a set of keywords and receive a bunch of items for sale relative to those keywords.

Like a lot of companies, eBay wanted to track and measure the success of its search functionality. Ben Foster, who was the company’s PM of Findings between 2001–2004, was in charge of that effort. In hindsight, he reflects about what happened:

“We ended up creating our own internal metrics [for evaluating search performance]. There are metrics that I think on the surface seem like they really make a lot of sense. So, we would look at the search results page and say ‘If it’s a successful search results page, then that should drive you to a bunch of item pages’… With the assumption being that if you click on 0 of those search results, then probably the search results page itself would have been a failure situation for the customer. So we thought the more view item pages we can get per search result, the better a job we are doing with the searches we are delivering.”

eBay’s search results quality metric

Search Results Quality = View item pages / Number of search results

To put this into perspective, imagine eBay’s strategy as the physical representation of having to go through all the aisles until you find the right item and reach the counter. In physical shops, you often have to, but the advantages of online is that you can go straight to the correct aisle, pick up your desired item and head to the cashier. Thus, having a faster, more pleasant and more fitting experience.

As a buyer going to a search results page, you would see the item cost for the given item. But since eBay was a platform powered by sellers, shipping costs would vary. What ended up happening is sellers to set artificially low prices in order to get clicks on their item page, only to jack up the shipping cost.

In essence, eBay’s sole focus on creating perceived value for sellers (more clicks on their item pages) was in direct confrontation with the needs of buyers.

The reluctance to solve the problem

Ben was aware of that problem, but as he explains he and his team @ eBay were reluctant to solve it. Since their primary metric for search results success was directly impacted by viewed item pages, it would be wrong to show display shipping costs on the search results page since it could drive down item clicks.

It took eBay a number of years to realise the magnitude of this miscalculation and change course. During that time, its buyers became increasingly frustrated with the experience they were getting out of the platform. By his own admission, this cost eBay a ton of loyal customers lost to its major competitor Amazon.

Going beyond the shipping cost mistake

The fact that buyers were misled by the price shown on the search results page was a considerable problem. But beyond that, the metric that eBay used in those early days was, in my opinion, fundamentally flawed. Why? Because the assumption that more item page clicks means better buyer experience is wrong.

Buyers on eBay, or Amazon, or any other similar platform are there because they have a need for an item, and they are looking to satisfy that need in the best possible way possible. In this definition, ‘best’ means the most pleasant and fast way achievable. They were not looking for pages to browse and click through, but instead for the item they want at their disposal.

So, are you saying that showing the item a buyer is most likely to purchase should have been the metric used?

No. But it is a step further.

If the eBay search results page shows as its principle result the item that the buyer is most likely to purchase, this is already a whole lot better than what eBay did originally. But purchase itself is not a guarantee of success.

How many times have you been dissatisfied with the quality of the item you received after purchasing online? How many times has the description not matched the actual product? Ah yes, now we’re on the same page now.

Those problems run much deeper than a metric as simple as the item pages / number of search results metrics.

Success for eBay is a buyer purchase that leads to a positive review, or at the very least no returns or follow-up complaints. Going there requires a completely different focus. An approach that focuses on the buyer at least as much as on the seller. In short, eBay had to learn that the problem it had to solve was not the number of item clicks. It was how to help the customer from the point of realisation (I want to buy this) to the point of usage (I am using my new item).

So, as we focus on SMART metrics, KPIs, OKRs or WTFs, bear in mind those are just pointers to help you get to the right mindset. They are not substitutes for thinking, immersion and empathy.

CREDITS

I want to give huge credits to the hosts of Rocketship.fm Michael Sacca and Mike Belsito for their fantastic podcast that inspired me to write this article. I also want to thank Ben Foster for sharing his experiences so openly with the world, so that we can learn from them. I also wish to thank Maarten Dalmijn for his useful comments in helping to make this article better!

Thank you for making it all the way down here. If you liked the article, please give it a clap or two… or 50!

Do you love quality writing? I do too. That’s why I created The Knowledgebase — a manually curated list of the very best articles online across product, design and development.

Thanks, till next time. 😉