AI and the Future of Radiology

AI Expert Panel: Paras Lakhani, MD, Ajay Kohli, MD, Mitchell Schnall, MD, PhD, Devang Gor, MD, MBA

Like other subspecialties in medicine, Radiology faces its own unique set of ongoing challenges. From decreasing reimbursement for radiology reports and procedures to increasing physician burnout, there are many pressing issues in the radiology field.

However, the most critical threat to radiology is considered to be the advent of deep learning and computer vision. The idea of having a black box, where a medical image is analyzed at one end and a full radiology report is produced at the other end, is now universally considered to be the most significant threat to the radiology profession. To best determine how radiology as a field should react to the emergent field of machine learning, the first step is to cultivate a better understanding of the fundamental technology. This is why leadership of the Pennsylvania Roentgen Ray Society (PRRS) decided to hold one of their events around artificial intelligence (AI) and the implications it holds for the future of radiology.

Related article: Artificial intelligence in radiology: Friend or foe?

It was fitting that the event to discuss the future of medical imaging was held at the historical Mütter Museum College of Physicians in Philadelphia, where the history and relics of medicine are showcased. The interest among the radiology community was quite palpable-all the seats were taken and the event was quickly standing room only.

The session was coordinated by the leadership of the Pennsylvania Roentgen Ray Society: Dr. Valeria Potigailo, Dr. Tessa Cook, Dr. Beverly Hershey, and Dr. Ryan Lee. Dr. Paras Lakhani, from Jefferson University Hospitals, delivered the keynote speech, “Strengths and Limitations of Deep Learning in Medical Imaging.” He provided an outstanding overview of the history of deep learning that led naturally into its applications within medical imaging. Dr. Lakhani’s talk masterfully captured the double-edged sword of deep learning in radiology, highlighting powerful applications and the inherent weaknesses for all such models.

As he discussed the current machine learning research for use cases in radiology, Dr. Lakhani no doubt simultaneously excited the data scientists and frightened the radiologists in the crowd. Particular neural networks are approaching human performance levels in detecting malignant or benign growths in images such as mammograms. For dealing with low quality CT image scans, different neural networks called autoencoders have demonstrated the ability to boost image quality by generating similar images with “repaired” pixel values, which the network has learned from training on similar data.

Finally, while the use of machine learning in radiology is still limited due to the relatively small number of high-quality, labeled datasets, the use of transfer learning opens up a near-term path forward. Neural networks that are trained on large, existing image datasets are able to “transfer” learned patterns (such as basic shapes, edges, etc.) readily to new datasets in more targeted fields, such as radiology. These transfer networks show dramatic increases in accuracy compared to networks trained on the target data alone.

Related article: Artificial Intelligence Makes Gains in Radiology

But as impressive as AI’s potential within radiology is, Dr. Lakhani highlighted several critical studies which lay bare the current weaknesses of these seemingly impressive models.

Researchers have shown the dangerous ability to cause neural networks to “hallucinate” by adding together two images. In a famous example, Google’s Cloud Vision system was tricked into guessing the prediction label “dog” on a clearly visible image of two men on skis, simply because a small number of pixels from a separate dog image were inserted into particular parts of the picture.

Related article: Artificial Intelligence: Own it Now

Similar disturbing results have arisen around the method of “backdoor poisoning attacks” on neural networks. By injecting a small number of specifically mislabeled training images into a training dataset, malicious actors can potentially insert “backdoors” into learning systems by tricking them into reliably predicting particular incorrect labels.

Perhaps most dangerous from a medical perspective, deep learning networks in general tend to be “black boxes,” which means they are difficult both to explain and to validate. It is quite common to train a neural network which seems to be performing incredibly well, only to discover that the network has been “cheating” by learning some artifact in the data. In one such example, a model learned to identify horses in images with near perfect precision. This amazing feat was accomplished simply because all such horse images in the training and testing sets contained a particular copyright tag! The clear technical vulnerabilities and lack of human-interpretable explanations for model predictions are strong evidence that AI is still far from replacing doctors; nonetheless, it can provide useful tools for skilled medical practitioners.

Following his talk, Dr. Ryan Lee took over to moderate the panel session on “Machine Learning and the Future of Radiology” which included Dr. Mitch Schnall, chairman of Penn Radiology, Dr. Devang Gor, chairman of Lehigh Valley Health Network, Dr. Paras Lakhani, assistant professor at Jefferson University Hospitals and Dr. Ajay Kohli, radiology resident physician at Drexel University College of Medicine. Many questions were addressed and there were three important takeaways:

1. Radiologists need to shape the conversation on the future of medical imaging. One of the most important takeaways was that the conversation around AI and radiology is being shaped by software engineers, data scientists and venture capitalists-who not only lack a complete understanding of the intricacies of the profession but also do not hold patient care paramount as do radiologist and clinicians.

Dr. Mitch Schnall made an excellent point when he noted that companies use healthcare data dating back decades. However, data in medicine is often not even relevant after just a few years because of rapidly evolving therapeutic and diagnostic tools. This is where medical experts will provide an integral part in the development of medical imaging technology-in order to apply technology effectively, it is imperative that developers understand the clinical relevance and significance of healthcare data.

Related article: Fear, Hope, and Hype For Artificial Intelligence

2. Understanding the technology and its limitations will be important for radiologists. Every radiologist has been in the position of having to make the call to differentiate an imaging artifact from true pathology (e.g., volume averaging artifact from true intracranial hemorrhage). This expertise acquired through training will not disappear with AI, but in fact will require an even higher level of understanding and experience of how deep learning technology works. It will therefore be imperative for radiologists to begin training with AI-assisted algorithms.

3. Deep Learning poses a threat and an opportunity. A concern was raised that deep learning can automate important tasks that are currently performed by radiologists. However, low hanging fruit for AI is in having some menial tasks of radiology-measuring changes of lung nodules, evaluating for changes in size of Multiple Sclerosis lesions-to be automated. Radiology has taken the lead in implementing many advanced forms of technology within healthcare. If the combined imaging workforce can embrace the adaptation of deep learning in medical imaging, then it can guide the implementation of deep learning in other subspecialties in medicine-such as using machine learning in screening colonoscopies or diagnosing electrocardiogram abnormalities for patients with cardiac disease, among others.

Related article: Q&A: AI For All Radiologists

The event was a success in providing an overview of deep learning technology. It highlighted current applications of deep learning technology in medical imaging, the strengths of these applications as well as their shortcomings. Most importantly, this event helped move the conversation of deep learning in medical imaging to a place where radiologists can shape its future. Yes, there are threats that deep learning poses to the profession of radiology. However, there are also immense opportunities for medical imaging and for all of healthcare.

Radiology will go through extensive change in the future as it becomes easier to implement deep learning algorithms in clinical care. However, the practice of radiology has embraced cutting edge advancements in healthcare and technology and through deep learning, radiologists can continue to embrace this change and shape the future of healthcare.

Max Henderson, PhD is a Senior Data Scientist that functions in a variety of roles at QxBranch, helping support projects involving quantum computation, machine learning, and data science. He has held Data Scientist positions at JPMorgan Chase & Co. and Lockheed Martin, providing technical solutions in cyber, bioinformatics, and natural language processing domains. Max has a PhD and MS from Drexel University in Physics as well as a BS in Physics from West Chester University. He can be reached through his Twitter: https://twitter.com/QxMax

Dr. Ajay Kohli is a resident physician in Radiology at Drexel University College of Medicine. He has completed an accelerated BS and MD program at Drexel University College of Medicine with clinical work at Kaiser Permanente in California. He has launched multiple entrepreneurial ventures as well as clinical studies within digital medicine which have gone on to win research competitions as well as grants (from using wearable technology in surgical oncology to using smartphones in heart failure and breast cancer management). He can be reached through his website: www.ajaykohlimd.com