Interactive workshops boost imagers' interpretive skills

Screening mammography in the U.S. has contributed mightily to the nearly 30% reduction in deaths from breast cancer since 1990.1 This success is due to many factors, including improvements in breast imaging technology, breast imaging teaching courses, the increased experience of a cadre of dedicated mammographers, and the implementation of the Mammography Quality Standards Act.2 The MSQA made a huge difference in the technical quality of the mammographic images but has never directly addressed the quality of interpretation.

The MQSA has fallen woefully short of achieving any significant assurance of quality. The minimum number of mammograms required to be interpreted annually is only 480, far less than the 3000 or more required by most countries with true organized mammography screening programs. Radiologists reading at such low volumes see only a few screening cancers a year, if they detect them. Audit and CME requirements under the MQSA are also weak.

The gaping disparity in MQSA regulation of mammography image quality and interpretation quality was highlighted in 2002 by articles in The New York Times regarding the poor quality of interpretation in the U.S.3 These reports gave rise to a growing concern among healthcare professionals, lay individuals, and legislators. Resulting investigations led to the publication in 2005 of an Institute of Medicine report, Improving breast imaging quality standards,4 that was highly critical of current mammography interpretation. It offered several recommendations: expansion of the superficial MQSA audit, establishment of centers of excellence, and introduction of demonstration projects and evaluations within those projects.

The report urged development of centers of excellence that could host interactive training programs in breast imaging. But it did not answer the question that those of us performing breast imaging face daily: What can we do to improve our interpretive skills now?

This is the same question I asked in 1986 at Dr. László Tabár's first mammography course in the U.S., in Los Angeles. Tabár challenged me to assimilate high-quality mammography techniques and interpretation into my practice in Albuquerque, read more films (at least 5000 per year), and audit my mammography practice, all of which I did.

By 1989, after eagerly adopting Tabár's interpretation techniques and performing a complete practice audit, my group and I found we had improved our cancer detection rate in the screening population by an astonishing 43%, while decreasing the size of the average invasive cancer we detected by 25%!5 Clearly, Tabár's teaching approach had resulted in measurable improvement in our interpretative skills. I was so taken by the success of his methodology that I resolved to establish a similar means to teach others.

INTERACTIVE WORKSHOPS

In assessing Tabár's teaching methods, I was struck by the interactive workshops he instituted to teach screening techniques. These used multiviewers filled with 100 or more cases, most of them normal but including a number of biopsy-proven moderately subtle invasive cancer cases (Figure 1). Tabár's idea was to create a simulated screening situation similar to most real-world practices.

Tabár personally reviewed every case, normal and abnormal, at the viewbox with small groups of about 15 attendees at a time, fielding questions regarding each case. Attendees could ask questions concerning any findings on the mammograms that bothered them, including the majority fraction of normal asymmetries and questionable distortions, while learning from Tabár how to construct templates in their minds for detecting the real lesions, the cancers that would otherwise go undetected.

The interactive nature of the workshop forced attendees to actively engage in a learning experience much more powerful than any passive method, such as didactic slide lectures, could provide. I found these workshops to be especially valuable in developing my expertise in mammography over the next few years and realized that others might benefit equally.

From this emerged a teaching paradigm that I felt could be used to improve interpretive skills for anyone willing to participate. This method had one major drawback, however. It was extremely labor-intensive. For that reason, Tabár soon switched to a slide format and eventually cut back on the screening portions of his courses, concentrating more on reaching larger audiences with his brilliant diagnostic teaching abilities.

I remained intrigued by the screening workshop concept. In the early 1990s, I established a professional link with two mammography soulmates, Dr. Robert Schmidt of the University of Chicago and Dr. Gillian Newstead of New York University. They had been equally impressed with Tabár's screening workshops and wished to incorporate this technique into their teaching arsenal. The three of us developed a set of 100 teaching cases, with slightly more than half being normal mammography cases and the others being proven cases of small invasive cancers, as well as a handful of calcification (ductal carcinoma in situ) cases.

We rolled out our first version of the screening workshop at a University of Chicago course in Palm Beach, FL, in 1997 and have since repeated the workshop 22 times, adding it to both my annual X-Ray Associates of New Mexico course in Santa Fe and to Newstead's spring course at NYU. The results have been impressive.

Because of its high cost and labor-intensive nature, the workshop is given only once annually, at a five-day breast imaging course in Santa Fe sponsored by X-Ray Associates of New Mexico. While it is the centerpiece, it is only one facet of the course: One day is devoted to breast MR, another to interventional procedures, and the rest to more traditional didactic sessions. This is, however, to my knowledge the only course in the U.S. currently offering such an intensive interactive screening workshop.

Attendees independently review 100 analog mammography cases during the initial portion of the course, at a suggested pace of one to two minutes per case. We have created a special scoring sheet for each attendee to fill out when reviewing each case (Figure 2), which acts as a reference when the subsequent supervised review occurs. The attendees are next divided into groups of four to six, and during a three-hour review session, each small group reviews the findings on each case with a faculty member.

The attendee group rotates to each of the three or four faculty members during the review session, with each faculty member reviewing a set fraction of the cases with each group. Group members are free to ask the faculty questions about each case, giving them the opportunity to benefit from the philosophies of multiple highly experienced and expert mammographers during the review session. We hope to improve the workshop at our Santa Fe course in August 2007 by converting to a digital format and allowing attendees to review on individual workstations.

Being able to identify up to 50 small cancers in a simulated screening situation offers a singular advantage. The sheer number of cancers viewed in a relatively short time frame acts to reinforce the characteristic features and variability of such cancers for the attendees, many of whom would normally not find that many small cancers in their own practices over the course of many years.

Attendees have found the workshops quite valuable. After our first Santa Fe workshop in 1997, an attendee called two days later to tell me he had just found a 6-mm cancer that he previously would never have recognized.6 At our course in August 2006, fully 40% of the attendees were repeaters, there to refresh their interpretive skills. Faculty members have been equally enthusiastic.

"The participants learn to screen with increased sensitivity and specificity and become much more comfortable at being faced with a large number of screening exams to interpret," said Dr. Richard Bird, a clinical assistant professor of radiology at the University of North Carolina, Chapel Hill.

Another faculty member, Dr. G. W. Eklund, a clinical professor of radiology at the Oregon Health Sciences University in Portland, reports consistently positive responses from attendees over the nine years he has taught at the workshops.

The screening workshops also produced an unexpected result. Even as attendees with good basic skills and motivation to improve got even better, those without such motivation and skills realized through their poor performance that perhaps they should not be interpreting screening mammograms. Some chose to stop reading screening studies in their own practices.

DIFFICULT CASES

We gleaned additional data related to interpretive skills testing and to the nature of cases found most difficult from a compilation of the answer sheets completed by some of the 1600 radiologists who have participated in the workshops since 1997.7 By creating a scatter plot of the performance of 100 individuals, we found a large variation in accuracy (Figure 3).

Even some attendees who interpreted high volumes of screening mammograms ( > 5000 annually) did rather poorly, whereas some who interpreted relatively low volumes ( < 2000 annually) did considerably better than average. Self-assessment of level of expertise was not reliably correlated with measured performance either, as many individuals who designated themselves "advanced" readers missed far more than the average number of cancers. These findings would seem to run counter to the European model for improving interpretation skills primarily by increasing case volumes, and they lend credence to the stance of many U.S. breast imaging experts that good education and reinforcement of interpretation skills are at least as important as case volumes, if not more.

Analysis of some of the difficult cases revealed situations that should prompt special caution in the interpreter. Most difficult is the second (synchronous) cancer in the same or opposite breast when a suspected cancer has been correctly detected (Figure 4). This problem, found in both dense and fatty breasts, was most likely due to so-called satisfaction of search. Tiny cancers within dense breast patterns also proved difficult and led to a large number of misses (Figure 5). Cancers seen on one view only were more frequently missed.

Our experience over the past 10 years has shown us the value of using an interactive workshop to teach and improve interpretive skills at the viewbox for radiologists of all skill levels. Now, as a result of the IOM report recommendations and various grants, the Breast Cancer Surveillance Consortium of the National Cancer Institute will be initiating a project that may extend our concept of the interactive workshop to a national level.8 The program, known as Assessing and Improving Mammography (AIM), has three components: first, an assessment of the effects of volume on interpretative performance; second, creation of an assessment test of screening mammography skills, to be given remotely in digital format; and third, application of one or the other of two teaching models. The first component is an in-person interactive teaching activity similar to our screening workshops. The second is a DVD version of the in-person model.

If successful, the AIM program should elevate the quality of mammography interpretive skills throughout the U.S. Breast imagers, however, must also adopt the other major recommendation of the IOM report, by performing a more expanded practice audit.9 In so doing, radiologists can raise the quality of breast imaging in this country.

Dr. Linver is director of mammography of X-Ray Associates of New Mexico and a clinical professor of radiology at the University of New Mexico School of Medicine in Albuquerque. He can be reached at MammoMike@aol.com.

References

1. Berry DA, Cronin KA, Plevritis SK, et al. Effect of screening and adjuvant therapy on mortality from breast cancer. NEJM 2005;353:1784-1792.

2. 21 CFA Part 16 and 900. Mammography quality standards; final rule. Federal register, Washington, DC: Government printing office, Oct. 28, 1997;62:55851-55994.

3. Moss M. Spotting breast cancer: doctors are weak link. The New York Times, June 27, 2002:1

4. Nass S, Ball J, eds. Improving breast imaging quality standards. Washington DC, Institute of Medicine, The National Academies Press, 2005.

5. Linver MN, Paster SB, Rosenberg RD, et al. Improvement in mammography interpretation skills in a community radiology practice after dedicated teaching courses: two year medical audit of 38,633 cases. Radiology 1992;184:39-43.

6. Miller JP. Personal communication, 1997.

7. Schmidt RA, Newstead GM, Linver MN, et al. Mammographic screening: sensitivity of general radiologists. In: Karssmeijer N et al, eds. Digital mammography Nijmegan 1998. Amsterdam: Kluwer, 1998:383-388.

8. Kerlikowske K. Novel collaboration advances research to improve mammography performance. SBI News Fall 2006:10-11.

9. Linver MN. The expanded mammography audit: its value in measuring and improving your performance. Sem Breast Dis 2005;8(1):35-42.