Can ChatGPT Provide Appropriate Information on Mammography and Other Breast Cancer Screening Topics?

Could ChatGPT emerge as a reliable source of information on breast cancer imaging for patients?

In a recent retrospective study, published in Radiology, researchers from tertiary care breast imaging departments posed 25 questions on breast cancer screening and prevention to ChatGPT, an artificial intelligence (AI) large language model chatbot.

The questions covered topics ranging from risks with family history of breast cancer and frequency of mammography screening to dense breasts and the meaning of scoring with the Breast Imaging-Reporting and Data System (BI-RADS). According to the study, the questions were submitted three times to ChatGPT and the responses were assessed by three fellowship-trained breast radiologists.

ChatGPT provided appropriate responses to 22 of the 25 questions (88 percent), according to the researchers.

“These findings suggest that ChatGPT holds great potential for automating provision of patient information about breast cancer prevention and screening, albeit with areas for improvement,” wrote Paul H. Yi, M.D., the director of the University of Maryland Medical Intelligent Imaging Center within the Department of Diagnostic Radiology and Nuclear Medicine at the University of Maryland School of Medicine in Baltimore, and colleagues.

(Editor’s note: For related content, see “Can ChatGPT Have an Impact in Radiology?”)

The authors noted a couple of caveats with ChatGPT, pointing out inconsistent responses to questions on breast cancer screening locations and breast cancer prevention, and an inappropriate response in regard to the timing of mammography scheduling after COVID-19 vaccination. While ChatGPT referenced American Cancer Society (ACS) guidelines to provide appropriate recommendations on screening mammography, Yi and colleagues noted there was no mention of mammography recommendations from the American College of Radiology (ACR) nor the United States Preventive Services Taskforce (USPSTF).

Accordingly, the study authors emphasized caution and physician oversight with the possible use of ChatGPT for patient-oriented information on breast cancer imaging.

In regard to study limitations, Yi and colleagues acknowledged that ChatGPT capabilities are rapidly evolving, and the ChatGPT version they utilized for the study was available in February 2023. In addition to noting that appropriateness assessments of ChatGPT are subjective, the study authors maintained that future research could assess how prompts to ChatGPT could influence or alter the chatbot’s medical advice recommendations.