Black women, women 61 to 70 years of age and women with extremely dense breasts may have a higher likelihood of false-positive results with mammography-based artificial intelligence (AI) software, according to findings from a new study.
For the retrospective study, recently published in Radiology, researchers examined the use of an AI algorithm (ProFound AI 3.0, iCAD) for assessing negative digital breast tomosynthesis (DBT) exams in 4,855 women (median age of 54). The study authors assessed case scores and risk scores with the AI-enabled software in a cohort comprised of White women (27 percent), Black women (28 percent), Asian women (28 percent) and Hispanic women (19 percent), according to the study.
The researchers found that false-positive case scores with the mammography-based AI software were 50 percent more likely in Black women and 30 percent less likely in Asian women in comparison to White women.
“If radiologists normalize the adoption of AI recommendations based on White patients — the largest demographic group in the United States —then they risk inappropriately higher recall rates for Black patients. This has the potential to worsen health care disparities and decrease the benefits of AI assistance,” wrote lead study author Derek L. Nguyen, M.D., a breast radiologist and assistant professor with the Department of Radiology at the Duke University School of Medicine in Durham, N.C., and colleagues.
Older women were also more likely to have false-positive case scores, according to the study authors, who noted a 3.5 odds ratio (OR) for women 61 to 70 years of age and a 1.9 OR for women 71 to 80 years of age in contrast to women 51 to 60 years of age.
While emphasizing that breast cancer incidence does increase with age, the researchers said the correlation between age and increased risk scores was nonetheless surprising given that case scores with the algorithm were solely based on imaging data.
“As patients age, there are some changes to mammographic appearance, but radiologists cannot easily predict a patient’s age based on mammogram appearance. Therefore, the algorithm may be detecting some underlying changes in the image that are driving differences in performance,” posited Nguyen and colleagues.
Three Key Takeaways
1) Higher false-positive rates in Black women. The study found that the AI algorithm for mammography had a 50 percent higher likelihood of false-positive results in Black women compared to White women. This suggests a potential bias in the AI software, highlighting the need for demographic-specific considerations in AI algorithm development and application.
2) Impact of age on false positives. Older women, particularly those aged 61 to 70, were more likely to receive false-positive scores from the AI software. This was surprising since the algorithm's case scores were based solely on imaging data. The researchers speculate that the AI might be detecting subtle, age-related changes in mammographic images.
3) Increased false positives with dense breasts. Women with extremely dense breasts were significantly more likely to have false-positive risk scores using the AI software, with an odds ratio of 2.8 compared to those with fatty breasts. This finding underscores the need for AI systems to account for breast density when evaluating mammography results to minimize false positives.
The study authors also pointed out that women with extremely dense breasts were significantly more likely to have false-positive risk scores with the AI software (2.8 OR) in comparison to those with breasts of fatty density.
While acknowledging challenges with a limited number of data sets available to develop AI technologies, the researchers said the Food and Drug Administration (FDA) currently doesn’t have a requirement for AI software to be tested on a demographically diverse population.
“Our findings suggest that vendors should provide greater granularity in the performance of their algorithms in specific patient populations,” noted Nguyen and colleagues. “The Food and Drug Administration should also develop guidelines for the demographic distribution of development data sets and greater transparency before approval.”
(Editor’s note: For related content, see “Mammography Study: AI Improves Breast Cancer Detection and Reduces Reading Time with DBT,” “GE HealthCare Launches AI Mammography Platform with Key Applications from iCAD” and “What a New Review Reveals About Mammography-Based AI and Breast Cancer Risk Assessment.”)
Beyond the inherent limitations of a single-center retrospective study, the authors noted that cases involving positive mammography results within two years were excluded from the study. The researchers added that they did not evaluate the cancer detection rate or sensitivity rate of the AI software in the study.
