This week, a highly anticipated ruling found that the Royal Free London NHS Trust broke the law when it gifted 1.6m patient-identifiable records to Google’s DeepMind, in November 2015.
DeepMind, an artificial intelligence company, states that it hasn’t used the records except as directed by the hospital trust. Using synthetic data, DeepMind has built an app, Streams, which will provide clinical alerts about kidney injury, and it has used some real patient data to test the app (it turns out, unlawfully) and, since January, deploy it. But the problem isn’t with patients who have a clinical need for kidney alerts. It is with everyone else, whose data DeepMind has now carefully structured, formatted and stored.
The ruling states that by transferring this data and using it for app testing, the Royal Free breached four data protection principles, as well as patient confidentiality under the common law. The transfer was not fair, transparent, lawful, necessary or proportionate. Patients wouldn’t have expected it, they weren’t told about it and their information rights weren’t available to them.
Rather than deleting the data or being fined, the hospital trust has signed an undertaking to clean up its act, and has three months to produce justifications for all data processing planned and in process.
Bold statements have accompanied the ruling. The Information Commissioner’s Office, which oversees data protection and led the investigation, emphasised it has “no desire to prevent or hamper” technical or clinical progress. But as commissioner Elizabeth Denham stated, “it’s not a choice between privacy or innovation”, and the legal breaches “were avoidable”. She concluded: “The price of innovation didn’t need to be the erosion of legally ensured fundamental privacy rights.”
These comments were echoed by the national data guardian, Fiona Caldicott. She clarified that the core issue “was not that innovation was taking place to help patients … it was the [inappropriate] legal basis used to share data which could identify more than 1.6 million patients to DeepMind”.
But these admirable statements, while certainly correct, are undermined by the fact that this sensitive dataset continues to sit on DeepMind servers, as it has for 20 months – and even though, on DeepMind’s own admission, it apparently doesn’t need it. This is all the more frustrating given there is a technological solution that involves both innovation and privacy: retaining the data in hospital trusts, and interfacing with apps such as Streams only when a clinical need arises.
Ever since DeepMind’s work with the NHS hit public consciousness, there have been two major issues in play. The first is what has always looked like an opportunistic data-grab of millions of detailed patient records. Regulators have now brought the issue home, but only halfway, since there’s no real remedy, and the information commissioner fought shy on the question of DeepMind’s joint culpability for the breaches, as well as the data windfall still in its hands.
It remains to be seen if the ruling is enough to deter the next data opportunist or if, in the end, it only deepens what scholars term the surveillance-innovation complex, where our bodies are “a source of presumptively raw materials that are there for the taking”.
The second issue is even more urgent. This is the seeming willingness of NHS trusts to embed a subsidiary of Google in the heart of the public health service, and what precautions are being taken for the long-term interests of patients.
Disappointingly, pressing questions around this subject were ignored by an independent nine-member panel that has enjoyed privileged access to DeepMind over the past year and issued its annual findings this week. The panel limited itself to recommendations about public engagement to address problems of “perception” about DeepMind’s ownership by Google.
Giving away our most sensitive and valuable data, for free, to a global giant, with completely uncertain future costs, is a decision of dramatic consequence. Yet crucial public conversation on this topic has been severely stunted. The Royal Free’s response this week, that it would forge ahead with DeepMind “to ensure the NHS does not get left behind”, does little to reassure that anything will change.
