Five out of 12 artificial intelligence (AI) products demonstrated higher than an 86 percent area under the receiver operating characteristic curve (AUC) for detecting tuberculosis (TB) on X-rays in a South African patient population with a high TB and human immunodeficiency virus (HIV) burden.
For the comparative study, recently published in the Lancet Digital Health, researchers assessed 12 computer-aided detection (CAD) software products for diagnosing TB on chest radiographs in 774 study participants, including 258 people who had bacteriologically positive TB. Twenty-six percent of the cohort had a history of TB and 18 percent of the study participants were HIV-positive, according to the study.
Insight CXR (Lunit) and Nexus CXR (Nexus) demonstrated the highest AUCs (90.2 percent and 89.7 percent respectively) for TB detection followed by qXR (Qure.ai) at 87.8 percent, JF CXR-2 (JF Healthcare) at 86.5 percent and ChestEye (Oxipit) at 86.1 percent.
Assessing the AI software modalities at 90 percent sensitivity thresholds and 70 percent specificity thresholds, the study authors noted similar results. At approximately 90 percent sensitivity, Insight CXR and Nexus CXR offered the highest specificity rates at 67.7 percent and 67.1 percent respectively. JF CXR-2 (62.7 percent), qXR (62.3 percent) and ChestEye (61.3 percent) had the next highest specificity rates at 90 percent sensitivity.
The researchers added that Insight CXR and Nexus CXR exhibited the highest sensitivity rates (89.5 percent and 88.8 percent) at the 70 percent threshold for specificity. They were followed by qXR (86.8 percent), JF-CXR-2 (86.4 percent) and ChestEye (86 percent), according to the study authors.
“Lunit, Nexus, JF CXR-2, and qXR maintained high sensitivity over a wider range of thresholds, resulting in more individuals being triaged by X-ray,” wrote lead study author Zhi Zhen Qin, MSc, who is affiliated with the Stop TB Partnership in Geneva, Switzerland, and the Department of Infectious Disease and Tropical Medicine at the Heidelberg University Hospital in Heidelberg, Germany, and colleagues.
Three Key Takeaways
- High accuracy of selected AI tools. Insight CXR (Lunit) and Nexus CXR (Nexus) demonstrated the highest accuracy in detecting tuberculosis (TB) on chest X-rays with AUCs of 90.2 percent and 89.7 percent respectively. These AI tools, along with qXR and JF CXR-2, maintained high sensitivity across different thresholds, suggesting their effectiveness for TB screening, especially in a population with a high burden of TB and HIV.
- Performance consistency across sensitivity and specificity thresholds. The AI software performed consistently well at both 90 percent sensitivity and 70 percent specificity thresholds. Insight CXR and Nexus CXR offered the highest specificity rates, indicating their reliability in reducing false positives in high-risk populations.
- Age and HIV impact on AI performance. The study found that sensitivity was higher in younger age groups (15–34 and 35–54 years) compared to those over 55 years. Despite lower AUCs in HIV-positive patients, the differences were not statistically significant, indicating that the AI tools may be generally effective for TB screening in this patient population.
While pointing out that HIV can impact how abnormalities are interpreted on chest radiographs, the researchers noted lower AUCs for all the reviewed AI products in this patient population but no statistically significant differences in comparison to those without HIV. However, they did notice key differences with respect to age.
“Sensitivity was statistically significantly higher in people aged 15–34 years and people aged 35–54 years than in people older than 55 years. Specificity was statistically significantly higher in people aged 15–34 years than in people aged 35–54 years and people older than 55 years,” pointed out Qin and colleagues.
(Editor’s note: For related content, see “Adjunctive AI Powered Tool for Tuberculosis Detection Gets FDA’s Breakthrough Device Designation,” “Study: Deep Learning System is Comparable to Radiologist Assessment of Chest X-Rays for Tuberculosis” and “Study Finds Four Out of Seven AI Algorithms Offer Better Lung Nodule Detection on X-Rays than Radiologists.”)
In regard to study limitations, the authors acknowledged possible inaccuracies with sub-analysis findings due to patient self-reporting of demographic, lifestyle, and clinical data. The researchers acknowledged that inclusion in the cohort was limited to those with symptoms of tuberculosis and abnormal X-rays. They also cautioned against extrapolation of the study findings to children as the study excluded patients under 15 years of age.
