Bad bots: how should doctors respond to untested technologies?

A CAUTIONARY TALE

The extraordinary rise and fall of Silicon Valley start-up Theranos is recounted in the unlikely page-turner Bad Blood.¹ Only a few years ago, Theranos, and its iconic founder and supposed genius Elizabeth Holmes, were celebrated for their game-changing technology that was set to revolutionise diagnosis. With its promise of machines that could perform hundreds of blood tests on a single drop of blood, Theranos was valued at $9 bn, while Holmes was fêted with fame, honours, and awards. Her board was stacked with dignitaries such as elder statesmen Henry Kissinger and George Shultz. Such luminaries, along with investors like Rupert Murdoch, were all captivated by Holmes’s vision of painless, accurate, and cheap diagnostics that would save lives and make them billions.

The most remarkable part of the story is, of course, that the technology never worked. But almost 1 000 000 blood tests were performed by Theranos before the scandal was exposed. So much of the ‘fake-it-until-you-make-it’ hyperbole in Bad Blood calls to mind another ‘game changer’, Babylon Health. Outlandish claims unaccompanied by peer review evidence, misleading advertising,² attempts to misrepresent³ and suppress critical regulatory outcomes,⁴ and an appetite for risk that most doctors would baulk at, run through the histories of both start-ups. Perhaps the most striking echo of Theranos is Secretary of State for Health Matt Hancock’s unconditional infatuation with Babylon’s GP at Hand, which he describes, contrary to the evidence,⁵ as one of the technologies ‘... which can improve patient outcomes, freeing up clinicians’ time and make every pound go further’.⁶

OUT OF HAND

No doubt GP at Hand delivers on its advertised promise to streamline antibiotic treatment for the sore-throated,⁷ but Babylon’s ‘symptom checker’ chatbot appears to remain a work in progress. The incredible claims that have been made for the app include the boast, made to great fanfare last year, that it outperforms human doctors in postgraduate GP examinations⁸ and a prediction made by Babylon’s founder Ali Parsa in 2018 that the app would be ‘an order of magnitude more accurate than any human doctor’ by the end of the year.⁹ Patients who were triaged using Babylon’s software in a 111 pilot were presumably unaware that the firm’s software is prone to suggest an ambulance for a simple dry cough but miss the classic symptoms of a heart attack.¹⁰ Meanwhile, Babylon’s puff about GP-beating accuracy have been dismissed, albeit rather courteously, in an assessment in the Lancet.¹¹

HOW SHOULD DOCTORS RESPOND?

Far from ushering in a great leap forward, the huge incentives to exaggerate benefits along with the limited capacity of regulators to reign in tech-chancers threatens a new dark age of waste and harm. When doctors try to sound the alarm we are dismissed as a conspiracy of luddites and ‘vested interests’. Politicians aren’t scientists: they tend to make decisions based on emotion and intuition without waiting for supporting evidence. So how can we be more persuasive?

Calling out unevidenced, misleading, or incorrect assertions is important but not enough. Referencing the Theranos story might help to temper some of the ‘fear of missing out’ with ‘fear of messing up’. Finding stories with an emotional ‘punch’ about the harms of under-tested technology is hard, but articulating, for example, indignation that desperately sick patients wait for hours for ambulances while we invest in hi-tech hotlines for the worried well is probably worth any number of statistics.¹²

But to help politicians and the public see through evidence-shy Svengalis, maybe we also need to get better at simplifying and explaining the story of medicine. In short, how constant iterative improvement (technology), demonstrated with proof (evidence), has transformed lives. Hopefully, in the end, our stories prove all the more persuasive for being true.