AI Algorithm Faster, More Accurate in Detecting COVID-19 on Chest X-rays

Northwestern University researchers have created an artificial intelligence (AI) platform that can outperform thoracic radiologists in the detection of COVID-19 on chest X-rays.

Although this new AI tool, called DeepCOVID-XR, is not intended to be used in place of RT-PCR testing, the investigators said, it can spot evidence of COVID-19 on these images 10 times faster and with 1 percent-to-6 percent greater accuracy than providers can.

Generated heatmaps appropriately highlighted abnormalities in the lung fields in those images accurately labeled as COVID-19 positive (A-C) in contrast to images which were accurately labeled as negative for COVID-19 (D). Intensity of colors on the heatmap correspond to features of the image that are important for prediction of COVID-19 positivity. Courtesy: Northwestern University

“We are not aiming to replace actual testing,” said senior study author Aggelos Katsaggelos, Ph.D., electrical and computer engineering professor who is also an AI expert with a courtesy appointment in the radiology department. “X-rays are routine, safe, and inexpensive. It would take seconds for our system to screen a patient and determine if that patient needs to be isolated.”

Rather, the intent, the team said, is to potentially identify patients with suspected viral infection earlier, prompting faster isolation to protect healthcare workers. It could also help detect patients who are not currently being assessed for COVID-19 who should be tested.

Related Content: AI Tool Scores COVID-19 Lung Infection Severity

The team will publish their study today in Radiology. The algorithm, which is still in the research phase, is publicly available for other institutions to train it with new data, the team said.

To create and train their algorithm, the team gathered 17,002 chest X-rays – the largest published dataset of chest X-rays used to train an AI system during the pandemic. Among those images were 5,445 from COVID-19-positive patients seen across several sites throughout the Northwestern Memorial Healthcare System.

The team pulled 300 random images from Lake Forest Hospital and compared the performance of DeepCOVID-XR against five experienced cardiothoracic fellowship-trained radiologists. Overall, it took each radiologist 2.5-to-3 hours to examine each set of images. But, the AI system was able to do it in 18 minutes. The tool also produced slightly better accuracy – 82 percent compared to the 76 percent-to-81 percent from the radiologists.

The team stressed that this performance came from specialty providers.

“These are experts who are sub-specialty trained in reading chest imaging,” said Ramsey Wehbe, M.D., a cardiologist and postdoctoral fellow in AI at the Northwestern Medicine Bluhm Cardiovascular Institute. “Whereas the majority of chest X-rays are read by general radiologists or initially interpreted by non-radiologists, such as the treating clinician. A lot of times decisions are made based off that initial interpretation.”

Using this AI algorithm could also be a more cost-effective and accessible way of identifying patients who should be isolated.

“Radiologists are expensive and not always available,” Katsaggelos said. “X-rays are inexpensive and already a common element of routine care. This could potentially save money and time – especially because timing is so critical when working with COVID-19.”

The team did point out that, while the AI algorithm can be more effective with patients who exhibit COVID-19 signs on their images, it cannot improve the identification of the number of patients who do not show signs. That shortcoming highlights the limits of the system.

“In those cases, the AI system will not flag the patient as positive,” Wehbe said. “But, neither would a radiologist. Clearly, there is a limit to radiologic diagnosis of COVID-19, which is why we wouldn’t use this to replace testing.”

For more coverage based on industry expert insights and research, subscribe to the Diagnostic Imaging e-Newsletter here.