Most likely, you’re familiar with Watson from the IBM computer system’s appearance on Jeopardy! in 2011, when it beat former champions Ken Jennings and Brad Rudder. Watson’s time on Jeopardy! was fun viewing, but it was also a very savvy public debut of a product that IBM wanted to sell: Watson Health.
Watson Health was supposed to change health care in a lot of important ways, by providing insight to oncologists about care for cancer patients, delivering insight to pharmaceutical companies about drug development, helping to match patients with clinical trials, and more. It sounded revolutionary, but it never really worked. Recently, Watson Health was, essentially, sold for parts: Francisco Partners, a private equity firm, bought some of Watson’s data and analytics products for what Bloomberg News said was more than $1 billion.
On Friday’s episode of What Next: TBD, I spoke with Casey Ross, technology correspondent for Stat News, who has been covering Watson Health for years, about how Watson went from being the future of health care to being sold for scraps. Our conversation has been edited and condensed for clarity.
Lizzie O’Leary: I look at the amount of money that went into pulling this together. Acquisition after acquisition. It was billions of dollars, and it sold for a billion in the end. Is there any way to read that as anything but a failure?
Casey Ross: Financially, certainly not. They spent way more money building this than they got back. Just the acquisitions alone cost them $5 billion. That it was sold so many years later, after so much in effort—7,000 employees at one point—means that this will as a total failure that they needed to just cut their losses and move on.
Why did IBM want to get into the health data business? What problem did they think Watson would help solve?
There’s a tremendous amount of information that is collected every day on the care of hundreds of millions of people. However, there is currently no way to connect that information, to link it to an individual across all the domains in which they get care, and then to develop a holistic picture of who they are, of what their diseases are, of what the best treatments are, and how to ensure that they get the best care at the lowest possible cost. There is no connectivity right now that can do that at scale. The people in the technology sector look at it and say, “This has to be fixed, and we’re going to fix it.”
Google, Microsoft, a lot of very big companies are extremely interested in health care. What is so attractive for these big tech companies about health care?
It’s one of the biggest parts of our economy. It’s a three trillion business that has legacy technology infrastructure that should be embarrassing. Tech companies are drawn to audacious challenges like this, and ones where they can make—if they’re successful—a ton of money.
That’s how things are today, but the same problems have been around since the advent of digitized data. In 2012, IBM closed a deal with Memorial Sloan Kettering, one of the preeminent cancer centers in the country, to train an AI to make treatment recommendations. What was the goal? What were they trying to do?
They were really trying to democratize the expertise of Memorial Sloan Kettering’s oncologists, to make that expertise available to patients all over the world and to develop this standardized engine for providing optimal treatment recommendations, customized to a patient, in front of a doctor, thousands of miles away. It was a beautiful notion. They were trying to say, “Well, let’s make it more objective. Let’s look at all of the data, and let’s tell every physician, for this patient in front of you, this is how they should be treated.”
So you get your biopsy results, and things don’t look good, but you’re not just getting the expertise or the biases of your particular oncologist. You’re getting the wealth of thousands of oncologists distilled into an algorithm?
Yes, you are getting all of that data, across so many different physicians, crunched down into a very digestible format and recommendation that could then lead to the best treatment for that patient.
Reading your reporting, it sounds like this was incredibly important to IBM. In 2015, Ginni Rometty, who was the CEO at the time, went on Charlie Rose. She said health care was “our moonshot.” How much of IBM’s hopes were hung on this thing?
The company made a huge bet that this could be the bridge to a different kind of future for IBM, which at the time was several years of quarterly revenue declines. They were trying to use Watson as a bridge to a different future where IBM wasn’t this old guard hardware company that everybody knew so well, but was operating on the cutting edge of artificial intelligence. Health care was the biggest, the buzziest use case. This was where they were going to really show the surpassing value of their technology.
To do that, IBM needed massive amounts of data on which to train Watson. It got that data through acquisitions, eventually spending some $5 billion buying a series of health data companies. What were those companies?
Truven, Phytel, Explorys and merge. Truven had the biggest insurance database in the nation with 300 million covered lives, Explorys provided a clinical data set of actual electronic health records kept by health systems representing about 50 million or so patients, Phytel added on top of that, and Merge had a huge imaging database. They had all this data and the idea was: Expose Watson to that, and it finds patterns that physicians and anyone else can’t possibly find when looking at that data, given all the variables in it.
Except that was not the reality. One of IBM’s high-profile partnerships with MD Anderson Cancer Center in Texas fell apart. A doctor involved said that there wasn’t enough data for the program to make good recommendations, and that Watson had trouble with the complexity of patient files. The partnership was later audited and shelved. What went wrong?
If you think about it, knowing what we know now or what we’ve learned through this, the notion that you’re going to take an artificial intelligence tool, expose it to data on patients who were cared for on the upper east side of Manhattan, and then use that information and the insights derived from it to treat patients in China, is ridiculous. You need to have representative data. The data from New York is just not going to generalize to different kinds of patients all the way across the world.
What was happening in a clinical setting? What was happening to patients?
Our window through the reporting was talking to physicians. We got concerns from them that the recommendations that it was giving were just not relevant. Maybe it would suggest a particular kind of treatment that wasn’t available in the locality in which it was making the recommendation, or the recommendation did not at all square with the treatment protocols that were in use at the local institution or, and more commonly so, especially in the U.S. and Europe, “you’re not telling me anything I don’t already know.” That was the big credibility gap for physicians. It was like, “Well duh. Yeah, I know that that’s the chemotherapy I should pursue. I know that this treatment follows that one.”
You got a hold of an internal IBM presentation from 2017 where a doctor at a hospital in Florida told the company this product was a piece of shit.
Seeing that written down in an internal document, which was circulated among IBM executives, was a shocking thing to see. It really underscored the extent of the gap between what IBM was saying in public and what was happening behind the scenes.
There were a lot of internal discussions, even a presentation, that indicated that the technology was not as far along as they’d hoped, that it wasn’t able to accomplish what they set out to accomplish in cancer care. There were probably a lot of people that believed, that truly did believe, that they would get there or that it was closer than maybe some people realized. I think the marketing got way ahead of the capabilities.
It’s very hard to listen to you and not think about Theranos, even though this is not a one-to-one parallel in any way. When you are trying to move by leaps and bounds with technology in the health care sector, it feels like a reminder that all things are not created equal, that making big leaps with people’s health is a much riskier proposition.
That underscores the central theme of this story: When you try to combine the bravado of the tech culture and the notion that you can achieve these huge audacious goals in a domain where you’re dealing with people’s lives and health and the most sacrosanct aspects of their existence and their bodies, you need to have evidence to back up that you can do what you say you can do.
Why did they continue on trying to rescue this product that they seemed to know internally was failing?
I think they had so much invested in it that it really was, for them, too big to fail. It had 7,000 employees. They’d invested so much time and energy on marketing in the success of the product that they really needed it to succeed.
Instead, they got a fail. But Watson’s fate certainly doesn’t mean that AI in health care is going away. Just recently, Microsoft and a large group of hospitals announced a coalition to develop AI solutions in health care. If you had to pin down a moral to the story, is it that AI in health care isn’t ready for prime time, or that IBM did it wrong?
I think it’s both of those. This will be a case study for business schools for decades. When you look at what IBM did and the strategy mistakes, the tactical errors that they made in pursuing this product, they made a lot of unforced errors here. It’s also true that the generation of technology that they had was nowhere near ready to accomplish the things that they set out to accomplish and promised that they could accomplish. I don’t think that the failure of Watson means that artificial intelligence isn’t ready to make significant improvements and changes in health care. I think it means the way that they approached it is a cautionary tale that lays out how not to do it.
Does the failure of Watson Health make you worry that it’s going to shut down other avenues for innovation? Will such a spectacular belly flop impede progress?
I don’t think so. There were so many mistakes that were made, that were learned from, that, if anything, it will facilitate faster learning and better decision making by other parties that are now poised to disrupt health care and make the progress that IBM failed to achieve. There’s a saying that pioneers often end up with arrows in their backs, and that’s what happened here. They’re an example, a spectacular example, of wrongheaded decision making and missteps that didn’t have to happen. By learning from that, I think advancement and progress and true benefits will be faster coming.
Future Tense is a partnership of Slate, New America, and Arizona State University that examines emerging technologies, public policy, and society.
