RSNA Challenge Opens Door for Brain Hemorrhage Machine Learning

Pinpointing a brain hemorrhage on a CT scan could get significantly easier in the near future thanks to the creation of a new database that has compiled the largest public collection of expert-annotated images collected to-date.

This dataset, stemming from the Radiological Society of North American (RSNA) Artificial Intelligence (AI) challenge, is the result of a collaboration between two medical societies (RSNA and the American Society of Neuroradiology (ASNR)) and more than 60 volunteer neuroradiologists. Details about the dataset were published today in Radiology: Artificial Intelligence.

The goal of both the challenge and the dataset, according to initiative leaders, is to accelerate the development of machine learning algorithms that can help detect and characterize these potentially life-threatening hemorrhages.

“The value of this challenge is to create a dataset that might lead to a generalizable solution, and the best way to do that is to train a model from data originating from multiple institutions that use a variety of CT scanners from various manufacturers, scanning protocols, and a heterogenous patient population,” said lead author Adam E. Flanders, M.D., a neuroradiologist and professor at Thomas Jefferson University Hospital. “In this case, we had data from three institutions and international participation. The dataset is unique, not only in terms of the volume of abnormal images, but also the heterogeneity of where they all came from.”

To complete this RSNA challenge, competition organizers built a dataset from the ground up, compiling brain hemorrhage CT datasets from three institutions: Stanford University in Palo Alto in California, Universidade Federal de São Paulo in São Paulo, Brazil, and Thomas Jefferson University Hospital in Philadelphia.

Both the RSNA and ASNR curated the dataset, and the ASNR issued a call for volunteers to annotate the images. In less than two days, the organizations had selected 60 volunteers to annotate 874,035 brain hemorrhage CT images in 25,312 unique exams. Each scan was marked as normal or abnormal, and abnormal ones were further detailed with hemorrhage subtype.

Upon the dataset’s release, Flanders said, interest was significant. More than 22,000 submissions were received from 1,787 individual competitors in 1,345 teams from 75 countries. Participation come from both inside and outside the medical realm.
“The 10 top solutions came from all over the world,” he said. “Some of the winners had absolutely no background in medical imaging.”

With a non-commercial license, the dataset is freely available to the AI research community for use and enhancement.
Based on the success of this dataset and collaboration with a subspecialty society, Flanders noted, next year’s competition for the characterization of pulmonary embolism on chest CT is being planned with the Society of Thoracic Radiology.