Can an emerging artificial intelligence (AI) triage platform facilitate improved workflow and detection for pulmonary embolism (PE) on computed tomography pulmonary angiography (CTPA) studies?
In a recent prospective study, published in Radiology, researchers compared radiologist assessment for PE in 503 study participants (mean age of 54) versus the use of adjunctive AI (Aidoc) in 1.023 study participants (mean age of 55.1).
For positive PE studies, the study authors noted a significant decrease in wait time for results with the use of adjunctive AI (11.3 minutes) in comparison to 21.5 minutes without AI. However, accuracy rates and miss rates for PE detection were similar with researchers noting a 97.6 percent accuracy rate and a 12.3 percent miss rate for radiologist assessment in comparison to 98.6 percent and 6.1 percent, respectively, for the use of adjunctive AI.
While the researchers noted a higher overall sensitivity rate for PE detection with adjunctive use of the AI triage platform (93.9 percent vs. 87.7 percent), they noted no significant differences for specificity rates (99.5 percent vs. 99.3 percent), positive predictive value (97.5 percent vs. 95.5 percent) and negative predictive value (98.8 percent vs. 97.9 percent) in comparison to radiologist assessment.
“ … The use of a Food and Drug Administration-cleared artificial intelligence (AI) triage system to detect any pulmonary embolism (PE) on CT pulmonary angiographic (CTPA) images did not result in aggregate improvement in radiologist accuracy, radiologist miss rate, or examination or report turnaround times,” wrote lead study author Steven Rothenberg, M.D., an assistant professor in the Cardiopulmonary Imaging Section within the Department of Radiology at the Heersink School of Medicine at the University of Alabama at Birmingham, and colleagues.
For overall imaging interpretation time and study turnaround time, the researchers noted no significant difference between the use of adjunctive AI (28.1 minutes and 190.3 minutes respectively) and radiologist assessment (28.9 minutes and 190.6 minutes respectively).
Three Key Takeaways
- Reduced wait times with AI triage. The use of adjunctive AI (Aidoc) with computed tomography pulmonary angiography (CTPA) scans significantly reduced wait times for positive pulmonary embolism (PE) results, from 21.5 minutes without AI to 11.3 minutes with AI. This suggests that AI can expedite the reporting process for PE cases, potentially improving patient care in urgent situations.
- Comparable accuracy rates. While AI reduced wait times, it did not significantly improve accuracy rates or specificity rates for PE detection compared to radiologist assessment. The accuracy rates for radiologists were 97.6 percent, while AI achieved 98.6 percent accuracy. This indicates that AI is not necessarily superior to human radiologists in terms of diagnostic accuracy for PE.
- Potential benefit for clinically insignificant PE. Adjunctive AI demonstrated a potential benefit in detecting clinically insignificant PE cases. It had a higher sensitivity rate (86.4 percent) and a lower miss rate (13.6 percent) compared to radiologist assessment (66.7 percent sensitivity and 33.3 percent miss rate) for these cases. This suggests that AI may be particularly useful in identifying less significant or subtle PE findings.
While they did note an overall reduced report turnaround time for adjunctive AI (45 minutes) in comparison to 53.7 minutes for radiologist assessment, the researchers found that removal of cases involving resident assessment led to similar report turnaround times with 39.6 minutes for adjunctive AI and 40.8 minutes for radiologist assessment without AI.
“ … Given that our average report turnaround time for dedicated PR examinations is less than 60 minutes, AI is unlikely to improve operation efficiency of the department for dedicated PR examinations,” noted Rothenberg and colleagues. “For practices that have longer service-level agreements or alternative patient care workflows (eg, PE response team), there may be a benefit to using this type of technology to improve turnaround times for positive examinations, and additional investigation is needed.”
The researchers did note benefits of the adjunctive AI platform for clinically insignificant PE. In comparison to radiologist assessment for these cases, adjunctive AI use had a nearly 20 percent higher sensitivity rate (86.4 percent vs. 66.7 percent) and a nearly 20 percent lower miss rate (13.6 percent vs. 33.3 percent), according to the study.
(Editor’s note: For related content, see “Can AI Improve Triage of CT Pulmonary Angiography Exams for Acute PE?,” “AI/CT Combination for Assessing Pulmonary Embolism Severity Gets FDA Nod” and “PE Triage Platform from RapidAI Receives FDA 510(k) Clearance.”)
In regard to study limitations, the authors conceded that findings from one tertiary care academic medical center limits broader extrapolation to other facilities. With a three-month gap between the two phases of the study, the researchers noted that factors such as staffing changes and updates to AI software could have factored into the results of the study. Other limitations included a lack of integration of the AI software with the worklist and questionable imaging quality for nearly 10 percent of the CTPA cases reviewed in the study.
