Imagers prove too hasty in dismissing breast CAD

Radiologists may be adept at rejecting hundreds of false positives flagged by computer-aided detection software, but they also have a strong tendency to dismiss correctly identified cancers, mistakenly believing the findings are benign, according to new research.

Radiologists dismissed 70% of correct CAD markings in a University of Chicago study, according to Dr. Robert Nishikawa, who presented the results at the 2006 RSNA meeting. He called for further research to explain the dismissive tendency.

The Chicago study involved 300 cases, including 66 patients with 69 cancers. Eight radiologists reviewed the studies and assigned a BI-RADS rating, then checked the findings of a CAD system that was state of the art in 2003 and regraded the studies.

They were twice as likely to reject correct prompts when the finding was visible in two views but marked by CAD on only one view. The cancers in these cases are often very subtle, and the radiologist may believe they are insignificant and do not need to be worked up. Cancers marked on both views tend to be more conspicuous, so they are more readily recognized when flagged by the computer.

"We need to improve [software] algorithms to have correlation in two views," said Nishikawa, an associate professor of radiology.

Consistent with other studies, the Chicago research showed that CAD increased sensitivity by about 10%, from 58% to 64%, and boosted recall rates by roughly the same amount.

"Radiologists' performance was higher when they used the computer," Nishikawa said.

If they would heed CAD's correct advice more often, radiologists could significantly reduce misses and boost sensitivity, he said. Performance would improve if CAD also played more of a diagnostic role by providing an indication of likelihood of malignancy. Such software has been developed and submitted to the FDA for approval.

Each case in the study averaged about two incorrect prompts, totaling 600 overall, most of which were correctly ignored. Of the 234 normal cases, CAD would have spurred a re-call of 85 patients, versus 76 without CAD.

In another study of CAD with full-field digital mammography, 65% of masses were marked on only one view, said Dr. Stamatia Destounis, who presented results at the RSNA meeting.

If radiologists see something on one mammographic view and are undecided, CAD may influence the decision against a workup, said Destounis, a radiologist at the Elizabeth Wende Breast Clinic in Rochester, NY. Consequently, radiologists must be aware that CAD sometimes marks cancers, particularly masses, in only one view, and they should not be too hasty in dismissing such findings. It is sometimes helpful in these cases to review prior mammograms or obtain an extra mammographic view.

Destounis' team evaluated CAD in 202 biopsy-proven lesions in 201 patients who underwent screening or diagnostic mammography. Overall, CAD picked up 74% of cancerous findings. It performed best in detection of calcifications, with 90% sensitivity and 75% of cases marked on two views. But CAD's abilities elsewhere fell short, as the software picked up only 61% of masses and 44% of cases of architectural distortion. Not surprisingly, performance varied depending on breast density, with sensitivity dropping from 79% in fatty breasts to 76% in heterodense breasts and 67% in dense breasts.

"In heterodense and dense breasts, CAD is much like the human eye," Destounis said.

To date, CAD in digital mammography has not been fully investigated.

"As use of digital mammography and CAD increases, further research is needed to improve breast cancer detection," Destounis said.

Some researchers, however, have serious doubts about CAD's value with conventional or digital mammograms.

"Resources are scarce, but wants are not. We need to prioritize and evaluate trade-offs and alternatives," said Dr. John Keen at the RSNA meeting.

CAD adds about $20 to $30 to the cost of the mammogram, a charge that is picked up by the patient or insurer. Keen, a radiologist at the John H. Stroger Jr. Hospital in Chicago, carried out a study that showed CAD is not cost-effective, based on price, mammography accuracy, cancer prevalence, and typical cancer detection rates with and without CAD. The reason for the exam (baseline or repeat mammogram) was also factored into his equation.

Keen found the following:

• CAD at baseline costs $1 million per life saved.

• For subsequent exams, CAD costs $1.6 million per life saved.

• Detection of additional invasive cancers costs $6 million per life saved.

It is important to note, however, that Keen assumed a mammography sensitivity of 89%, which is much higher than typical rates in general practice. This figure came from experience in the Breast Cancer Surveillance Consortium. Keen's assumption proved controversial with some RSNA attendees, because CAD's value is thought to be inversely proportional to radiologist performance, and assuming lower sensitivity could improve the cost-effectiveness profile.

Keen also pointed out that CAD picks up more ductal carcinoma in situ.

"CAD is not likely to be cost-effective from society's perspective, especially when accounting for DCIS overdiagnosis and overtreatment. We need to look and see if CAD is an expensive magnifying glass to identify DCIS," Keen said.