What’s the Best Way to Test Radiology Residents?

Depending upon when you finished residency, there’s a good chance your board certification exam looks different from some of your colleagues. The current question in the industry, though, is whether the changes to the test have been a good idea.

Roughly a decade ago, the American Board of Radiology (ABR) eliminated the oral portion of the board exam, making it a multiple-choice test. While many in the industry support the move toward more psychometric testing, a standardized method that determines a resident’s mastery of knowledge, many worry about the move away from testing radiology’s more interactive skills.

“The oral boards were a hot-seat conference where you saw a case, gave your interpretation, and answered questions about what you were saying,” says Lincoln Berland, MD, emeritus professor of radiology at University of Alabama at Birmingham. “It was the way for the group to determine how well you were with skills you would use in a real-life environment.”

Berland recently published an article in the Journal of the American College of Radiology that discusses the validity and effects of psychometric testing in board certification. In it, he calls for the ABR to work with national education groups and experts to modify the multiple-choice format to include a more real-world skills assessment.

The Case Against Psychometric Testing

Although multiple choice exams can provide a measure of how well residents have retained medical knowledge throughout their training, there are many reasons why this style of testing is inadequate for determining a new radiologist’s clinical capabilities, Berland says.

First, it mimics the testing tactics used throughout K-12 education, he says. In these instances, students are expected to have proficiency in topics covered on the standardized exam, so teachers spend more time providing instruction in those areas. Consequently, some areas fundamental to successful radiology practice, such as communication, can be overlooked.

Second, in order to prepare for today’s board certification test, residents largely retreat from clinical interactions to study. The time spent reviewing didactics is helpful, but it eliminates the opportunity to hone interpersonal skills.

“Residents basically check out for months at a time to study for the exam,” Berland says. “They should be learning the skills for interacting with patients and other physicians. Instead, they’re allowed to stay out of circulation and learn how to answer multiple-choice questions.”

Third, relying on a multiple-choice exam also denies the certification board the opportunity to see how a resident produces radiology’s main work product-the report. It also doesn’t assess how well he or she can access medical records, how well they merge and analyze information, or how innovative and creative they can be with patient care.

“These things don’t get emphasized,” he says. “And, that’s why we still have problems with the quality of radiology reporting.”

The Importance of Communication

Without a doubt, learning proper communication skills is a vital part of residency training, says Seth Hardy, MD, a radiologist with Foundation Radiology Group in Pennsylvania. It’s the critical skill that keeps the industry sustainable.

“No one gets into radiology because they’re not smart. But, the ability to convey that value, that intelligence, in a meaningful way is literally where the rubber meets the road,” Hardy says. “We’re all like these Ferrari race cars, but if you’re not able to communicate well, it’s like putting cheap tires on a really expensive Italian car.”

Mastering communication techniques is critical, he says, because it can help you side-step problems once you’re out in practice.

Being able to construct clear, direct radiology reports is of paramount importance because all of your reports must be actionable. You must be able to create a work product that is accessible to primary care providers and nurse practitioners without prompting any confusion.

“If you’re not clear in your report, it can result in extra phone calls to other ‘go-to’ radiologists who can translate what you’ve submitted,” Hardy says. “There’s a potential for a little bit of double-duty for the ‘go-to’ radiologist. It could also produce an unintended, but unavoidable, delay in care.”

Good communication skills also ensure you’re delivering the right message to the referring physician rather than simply transmitting a cut-and-dry diagnosis. Merely identifying the problem doesn’t always give your colleagues the direction needed to take next steps.

“If a referring doctor sends a patient in with a specific question, and you don’t answer it, your report is worthless,” he says. “You’re not adding value.”

And, a multiple-choice test isn’t an accurate way to determine whether you’re adequately answering the clinical question. A psychometric test that only concentrates of rote medical knowledge, he says, doesn’t have the bandwidth to accurately assess your efficacy or the value you bring to the table.

The ABR’s Perspective

Although Berland and others in the industry are supportive of the return of some form of the oral exam, ABR leaders argue assessing the skills observed during the oral exam were never the organization’s responsibility.

“It’s never been the intent of the ABR exam to assess communication and reporting or interacting with patients,” says Valerie Jackson, MD, ABR executive director and president-elect of the Radiological Society of North America. “That’s been the domain of the residency program. Only the residency program can truly assess those soft skills.”

Because resident program directors decide which residents are ready to sit for the board exam, she says, they’re in the best position to analyze how well a particular resident has developed the interpersonal skills used in daily practice.

The ABR certification exam, she says, is intended only to test content knowledge. In fact, today’s exam offers a second part that can test a candidate on his or her content knowledge in a chosen sub-specialty area.

In addition to the test’s focus, Jackson adds, the price tag for administering the oral exam was high, and the cost was divided and passed down to individual candidates. Removing the oral exam from the certification process helped keep costs lower and more affordable for residents pursuing certification.

Overall, though, she says, conducting an oral exam wasn’t efficient. It slowed down the assessment process when there’s a lot of knowledge on which to test a resident.

“You might only get through a handful of cases during the session, depending upon how the candidate was doing,” she says. “It doesn’t exactly allow you to do a robust analysis. You might have roughly 60 items you need to assess, and you’d be lucky to get through five or six with some candidates.”

Still, Berland says, completely omitting the evaluation of the skills radiologists use daily that can’t be taught in books undermines the goal of determining which residents are qualified for board certification. Book knowledge isn’t enough to demonstrate that a radiologist adds value in today’s healthcare environment.

“You wouldn’t give a performer on television or on stage an award based on who memorized their lines the best,” Berland says. “Basically, that’s what we’re doing. We’re drilling people without determining how they can apply what they’ve learned in the real world.”