Google DeepMind's NHS deal under scrutiny

Google DeepMind's NHS deal under scrutiny

A deal between Google's artificial intelligence firm DeepMind and the UK's NHS had serious "inadequacies", an academic paper has suggested.

More than a million patient records were shared with DeepMind to build an app to alert doctors about patients at risk of acute kidney injury (AKI).

The authors said that it was "inexcusable" patients were not told how their data would be used.

Google's DeepMind said that the report contained "major errors".

It told the BBC that it was commissioning its own analysis and rebuttal, which the authors said they welcomed.

When the deal between London's Royal Free Hospital and DeepMind became public in February 2016, some three months after the data started to be collected, it caused controversy over the amount of patient information being shared and the lack of public consultation.

Hal Hodson, a former New Scientist journalist, and co-author Julia Powles, a Cambridge University academic, said there are still big questions to be answered about the tie-up.

"Why DeepMind, an artificial intelligence company wholly owned by data mining and advertising giant Google, was a good choice to build an app that functions primarily as a data-integrating user interface, has never been adequately explained by either DeepMind or Royal Free," they wrote.

The app contains no artificial intelligence although DeepMind has said that it was hoping to incorporate AI techniques to create smarter alerts in future.

The criticisms in the paper included:

In response, DeepMind and the Royal Free issued a joint statement: "This paper completely misrepresents the reality of how the NHS uses technology to process data.

"It makes a series of significant factual and analytical errors, assuming that this kind of data agreement is unprecedented.

"In fact, every trust in the country uses IT systems to help clinicians access current and historic information about patients, under the same legal and regulatory regime."

The authors invited both to respond in an "open forum", adding: "The obvious fact is that we care about Google and DeepMind getting into healthcare because it is a break from the norm.

"These companies are entirely different to specialised health IT and infrastructure providers, and the sweeping analogy does a disservice to the public."

The NHS does have information-sharing agreements with a range of third-party firms, but this is the first such deal with a major US tech firm.

DeepMind's initial assertion that the NHS had 1,500 other agreements with third-party organisations that process patient data has since been described by the NHS as "inaccurate". There is no central database on how many there are, the BBC was told.

The app is currently the subject of an investigation by the Information Commissioner's Office while the National Data Guardian, tasked with safeguarding health data, is also looking at it.

In a statement, the ICO told the BBC: "Our investigation into the sharing of patient information between the Royal Free NHS Trust and Deep Mind is close to conclusion.

"We continue to work with the National Data Guardian and have been in regular contact with the Royal Free and Deep Mind who have provided information about the development of the Streams app.

"This has been subject to detailed review as part of our investigation. It's the responsibility of businesses and organisations to comply with data protection law."

The National Data Guardian added: "Our consideration of this matter has required a thorough approach in which the NDG and her panel have kept patients' rightful expectations of both good care and confidentiality at the forefront of discussions.

"The NDG has provided a view on this matter to assist the ICO's investigation and looks forward to this being concluded as soon as practicable."

DeepMind has been at pains to make clear that none of the data collected for the app has been shared with parent company Google.

AKI is a serious condition, linked to 40,000 deaths a year in the UK and leading to a range of other health issues from minor kidney dysfunction to the need for dialysis and transplant.

In February, DeepMind published details about how the app was improving patient care.

It revealed that more than 26 doctors and nurses at the Royal Free are now using Streams and that each day it alerts them to 11 patients at risk of AKI.

Sarah Stanley, a consultant nurse who leads the patients at risk and resuscitation team, said: "Streams is saving us a substantial amount of time every day. The instant alerts about some of our most vulnerable patients mean we can get the right care to the right patients much more quickly."

DeepMind has acknowledged that it could have done better in the way it engaged with patients whose data was being used and, on the back of the criticism, agreed to set up patient forums.

It published a strategy on future patient engagement which opens by saying: "Outcomes are better when patients and clinicians make decisions together."

Add Comment

all comments

  Other news

Inmarsat's European short-haul wifi spacecraft launches

Inmarsat's European short-haul wifi spacecraft launches..

28-Jun, 21:19

A new Europe-wide wifi service for aeroplanes came a step closer on...

British Army too small, warn US military experts

British Army too small, warn US military experts..

28-Jun, 12:48

The UK should not make further cuts to its military budgets because its...

Celine Dion turned heads in Paris as she stepped out in a ?6,000 Balmain snakeskin coat and a band T-shirt

Celine Dion turned heads in Paris as she stepped out in a ?6,000 Balmain snakeskin coat and a..

28-Jun, 15:54 / Read more about: Celine Dion fashion news...

Dixons Carphone reports record full-year profits

Dixons Carphone reports record full-year profits..

28-Jun, 03:37

Dixons Carphone's annual profit has surpassed ?500m for the first time,...

Last-minute bargaining keyed passage of GOP health care bill

Last-minute bargaining keyed passage of GOP health care bill..

05-May, 03:26

President Donald Trump was ready to celebrate. On the day House...

Congo court sentences 9 rebels to death over killings

Congo court sentences 9 rebels to death over killings..

10-Mar, 11:14

A Congolese court on Friday sentenced to death nine rebels and acquitted...

Philippine leader names his senator ally new foreign chief

Philippine leader names his senator ally new foreign chief..

10-May, 22:17

A Philippine senator who is a staunch defender of President Rodrigo...

Failure on health bill also hurts prospects for tax overhaul

Failure on health bill also hurts prospects for tax overhaul..

26-Mar, 22:58

House Republicans' failure to repeal Barack Obama's health care law deals...