Emerging research suggests an adjunctive artificial intelligence (AI) software for head computed tomography (CT) scans shows no significant impact in detecting intracranial hemorrhage (ICH) or improving report turnaround times.
For the prospective study, recently published in the American Journal of Roentgenology, researchers evaluated the adjunctive use of a commercial AI triage software in 9,954 non-contrast head CT exams for a total of 7,371 patients (mean age of 54.8). The first phase of the trial was comprised of 3,716 CT exams and 735 positive ICH diagnoses and the second phase included 6,238 exams and 1,368 positive ICH diagnoses, according to the study authors.
The researchers found that the AI software demonstrated no significant difference in comparison to unassisted radiologists with respect to the accuracy rate (99.2 percent vs. 99.5 percent), sensitivity rate (98.9 percent vs. 98.6 percent) and positive predictive value (99.7 percent vs. 99 percent).
Unassisted radiologists also had a higher specificity rate (99.8 percent vs. 99.3 percent) than adjunctive use of the AI software, according to the study authors.
“Diagnostic performance was not significantly different between radiologists without and with AI in subanalyses stratifying by examination timing (initial vs. follow-up examinations) or interpreting radiologist primary appointment (neuroradiologists vs. emergency radiologists). The results overall fail to support use of AI assistance for ICH detection on head NCCT (non-contrast CT) examinations,” wrote lead study author Cody Savage, M.D., who is affiliated with the University of Maryland Medical Intelligent Imaging (UM2ii) Center in the Department of Diagnostic Radiology and Nuclear Medicine at the University of Maryland School of Medicine, and colleagues.
Three Key Takeaways
1. Limited Impact of AI on ICH detection. The study found no significant difference in diagnostic accuracy, sensitivity, or positive predictive value between radiologists using adjunctive AI software and those working without it in detecting intracranial hemorrhage (ICH) on non-contrast head CT scans.
2. Higher specificity without AI. Unassisted radiologists demonstrated a slightly higher specificity rate compared to those using the AI software (99.8 percent vs. 99.3 percent), suggesting that AI may not improve specificity in clinical practice.
3. No improvement in turnaround time. The use of AI did not improve report turnaround times for ICH-positive CT exams, with unassisted radiologists actually showing a shorter mean turnaround time (147.1 minutes vs. 149.9 minutes).
The study authors also pointed out that unassisted radiologists had a shorter mean report turnaround time for ICH-positive CT exams in comparison to the use of adjunctive AI (147.1 minutes vs. 149.9 minutes).
“This finding is important as the rapid identification of the presence (or lack) of ICH guides early treatment decisions. Such decisions may require diagnosis within a specific time frame in order to initiate certain treatments (e.g., thrombolytic therapy of ischemic infarct) that may improve survival or mitigate disability,” emphasized Savage and colleagues.
(Editor’s note: For related content, see “What a Meta-Analysis Reveals About Cone-Beam CT for Diagnosing Acute Intracranial Hemorrhage,” “FDA Clears AI Advance for Detecting Intracranial Hemorrhage on Non-Contrast CT” and “FDA Clears AI Software for Assessing Intracerebral Hemorrhage on Non-Contrast CT.”)
Beyond the inherent limitations of a single-center study, the authors acknowledged the lack of a randomized controlled study design. They also acknowledged that reevaluation of CT head exams was primarily limited to cases involving discrepancies between AI interpretation and radiology reports. The researchers noted that the AI widget interface was not integrated with the reading worklist. They also pointed out that the 2021 data from the study may not reflect subsequent updates to the AI algorithm.
