Medical imaging AI fails to meet expectations

Jun 16, 2021

Image: Unsplash

The background

Just a few years ago, most experts agreed that algorithms would dominate radiology until 2021. Medical imaging was considered the predestined field of application for AI, since it involves pattern recognition in large amounts of data.

Turing Award winner Geoffrey Hinton said in 2016: “We should stop training radiologists now. It’s just completely obvious that within five years, deep learning is going to do better than radiologists.” And he was not alone in his assessment. According to a 2018 survey of 322 Canadian medical students, 68 percent believed AI would reduce the demand for radiologists.

The status quo

Fast-forward to 2021, and more than 80 AI algorithms for medical imaging have been approved by the FDA (and a similar number in Europe). However, the reality about these algorithms’ utilization is sobering. Only about 11 percent of radiologists use AI for image interpretation in clinical practice, and 72 percent of those not using AI have no plans to do so. The reason for this slow adoption is poor performance. Only 5.7 percent of users report that AI always works, while 94 percent report inconsistent results.

So, what’s holding AI back? According to AI guru Andrew Ng, the reason is a proof-of-concept-to-production gap that AI not only has in healthcare, but in all types of application fields. It is not unusual for an algorithm to work perfectly in one hospital but then produce completely worthless results in another one due to a slightly different imaging protocol.

Our perspective

A rule of thumb in artificial intelligence is that for every euro you put into developing an algorithm, you have to put another 100 euros into deploying and supporting it. In the medical field especially, we need to be more realistic about our expectations for AI.

The FDA should give greater consideration to the robustness and transferability of algorithms during the approval process. Unlike medical devices, AI algorithms are extremely sensitive to variations in operating conditions.