What to Expect From the New Diagnostic Radiology Core Exam

Eleven-hundred radiology residents are about to take the most comprehensive test of their lives. Thirteen and a half hours. Sixteen subspecialty categories plus physics and quality. Hundreds of questions. “Everyone is very scared of the test,” said Shahrouz Tahvilian, MD, a third year radiology resident at Monmouth Medical Center in New Jersey.

It used to be that the oral exams, taken at the end of the resident’s fourth year, were the highly anticipated (and feared) event. But the 2010 residency class was the first to transition to the new exam structure, and will be the first residency class to finish without board certification. The new testing paradigm is an all-encompassing core exam at the end of the third year, and a more specialized certifying exam 15 months after finishing residency. This year’s core exam, given in June to the 2011 class, will be the second time the core exam is administered.

What Can Residents Expect?
The image-rich computerized exam is comprehensive, covering the entire field of diagnostic radiology. It’s a combination of the previous physics, clinical and oral exam, said Kay Vydareny, MD, associate executive director of the American Board of Radiology (ABR) for diagnostic radiology and the subspecialties.

As part of the group taking the first core exam last October, Justin Bigger, MD, a chief resident at Penn State Hershey Medical Center, thought it was comprehensive but fair. He estimated that residents are presented with 300–400 cases per testing day, some with several questions each. In all, Bigger said, he probably answered 500–600 questions each day of testing.

The two-day test is given in June and October. Each testing day is 6–7.5 hours and covers 18 categories, including 16 subspecialties like musculoskeletal, cardiac, breast, ultrasound, CT, MRI and pediatric radiology, along with quality and safety, and physics. The physics questions are integrated into the test, but scored separately. Those who fail one to five categories condition the exam, and must repeat them. Failing more than five sections requires the entire test to be retaken.

The test is offered in the Chicago and Tucson ABR exam centers. Since the June test group is large (1,100 residents), they offer it twice, a few days apart. While the tests for the two sessions are very similar, they aren’t identical, Vydareny said.

Test questions are written by volunteer committees of radiologists from academia, residency program faculty and private practice. Committee members find classic cases that illustrate particular pathologies, said Michael A. Bruno, MD, professor of radiology and medicine, and director of quality services and patient safety at Penn State Hershey Medical Center, the committee chair for writing quality and safety questions for the core exam.

About half of the questions are retained from the previous exam, said Bruno. “We’re writing hundreds of new questions,” he said. All the questions are analyzed for their performance. “If a large number of candidates miss a question, the problem is probably the question.”

As a criterion-referenced exam, said Vydareny, the volunteer committee assembling the test decides the passing standard, and theoretically 100 percent could pass since it’s not graded on a curve.

While it’s a computerized exam and, in theory, the residents could get instantaneous results, the ABR reviews the questions after testing to see which are psychometrically valid. She acknowledged that previous test takers waited a long time for results, and said they have shortened that time period, but residents still may not get results for eight weeks.

Study Time
For the previous oral exams, residency programs often gave residents a lot of time off to study. That’s changed. The study time off for third year residents varies, said Bruno. “At Penn State Hershey, we give them two hours at the end of the day to study. It’s not as much as it used to be, but they probably don’t need it as much.”

Having completed the test in October, Bigger said that his classmates all spent a similar amount of time studying, and all passed. After a normal work day, the faculty offered physics and subspecialty reviews to the group four nights a week from 5:00–6:30. After going home, residents spent at least another hour or two of self-directed study, plus more on the weekend.

At Michigan’s Henry Ford Health System, third year residents work 9–10 hours a day on lighter rotation, with call shifts completed earlier in the year, versus 14 hours for the other residents with call, said Lisa Walker, MD, a third year radiology resident. “We take that time at home to study,” she said, adding that her program offers a number of review and educational sessions, plus additional workstation teaching.

How to Study
The ABR posts a study guide online with potential topics covered. “That list is exhaustive,” said Bigger. That said, there were no surprise questions during the exam. While some were harder than others, “I didn’t think any of the questions were unfair,” he added.Sentiments like that have been helpful to Walker, who is taking the test in June. “Based on the guide they’ve put up, it seems like an enormous quantity to learn, and some is very specific,” she said, noting that those who took the test before her said the images and questions aren’t tricky or obscure.

[[{"type":"media","view_mode":"media_crop","fid":"25080","attributes":{"alt":"","class":"media-image media-image-right","id":"media_crop_5810062536038","media_crop_h":"0","media_crop_image_style":"-1","media_crop_instance":"2246","media_crop_rotate":"0","media_crop_scale_h":"0","media_crop_scale_w":"0","media_crop_w":"0","media_crop_x":"0","media_crop_y":"0","style":"height: 200px; width: 299px; border-width: 0px; border-style: solid; margin: 1px; float: right;","title":" ","typeof":"foaf:Image"}}]]

The ABR study guides list the entire domain of a category, like what could be included under GI, said Vydareny. The quality and safety portion has a 40 page study guide, doubling as a syllabus and lesson plan, which “covers all of the topics [residents] need to know for the quality and safety portion of the exam,” Bruno said. “If they learn all of the topics discussed there, they should pass [that portion].” Not every residency program teaches the quality and safety issues covered in the test, so the guide is the only training resource some residents will have in that area, he said.

There’s a practice exam on the ABR website. After Vydareny heard comments that the practice exam questions were very different from the core exam, she reviewed the two and disagreed. The questions were made from the same group of items and were very similar, she said. Going through the practice exam is helpful not only for content, but in helping the resident get used to the testing interface.

The best way to prepare for the exam, said Bigger, is to study the material from the first three years of the residency program. Then several months before the test, “really hit the review questions hard.”

In addition to the ABR materials, residents interviewed by Diagnostic Imaging used a variety of resources including RADprimer, RSNA modules for physics, RadCases, Face the Core and boardvitals.com, in addition to outside review courses and taking case-base imaging review courses at meetings like ARRS.

Since clinical questions are centered on particular cases, “we tell the residents it behooves them to see as many different cases as possible,” Bruno said. Do a broad case-based review of musculoskeletal reads rather than memorizing lots of little facts, for example.

After the first test in October, residency programs adapted their education to better prepare residents for the exam. For example, the fourth year residents in Walker’s program felt less prepared for the physics portion, so the residency program strengthened that element for her class. In addition, faculty started changing from case-based conferences to quicker cases, modifying and adding multiple choice type questions.

How it Went Last Year
While the core exam was stressful for those in the initial test group, the ABR had an additional concern: could they successfully deliver a computer-based exam to a large number of testers in their exam centers? It went perfectly, said Vydareny. And the passing results were very similar to the oral exam. About 87 percent of residents passed, 11.5 percent failed and the rest conditioned it. Those conditioning it were all in physics, which Vydareny said was reasonable because the passing standard was higher in physics than the other categories. Also, people who were weak in multiple radiology categories failed the entire exam rather than conditioning it.

Why the Change?
Some radiologists still question whether the new testing system is a positive change. The decision to change was based on many things, said Vydareny, but predominantly it was because the practice of diagnostic radiology is increasingly subspecialized. The ABR trustees wanted an exam paradigm that better reflected this subspecialization. This also frees up time in the fourth year, so the residents would spend their time learning one or more subspecialties rather than spending the bulk of the year studying.

The move from oral to computer-based exams had several other benefits. The oral exams were not valid or reliable because no two exams were the same, said Vydareny. By using half of the same questions for each version of the new core exam, and checking them after testing, they can validate that the test was fair and accurate.  

The oral exam also had a strong psychological component, said Vydareny, with some residents so fearful and anxious during testing that they couldn’t perform normally.

Testing Concerns
The change in timing and structure caused controversy, and those affected by the change aren’t sure of the implications. “Personally I feel like we’re in a bubble year,” said Tahvilian. Since much of the test is multiple choice, some residents are concerned that it doesn’t test the actual skills a diagnostic radiologist needs. “When you see a case, the patient isn’t going to have options with the diagnosis. You have to be able to take a differential and, from your knowledge, list the differential in the order that it is. This test might be taking away from that. No one really knows.”

While the majority of the exam is multiple choice, said Vydareny, there are other types of questions, including labeling anatomic structures, and matching diagnoses to images. They’re hoping to continue adding new question types that don’t just test fact-based recall, but rather knowledge synthesis and management decisions, a part of the previous oral exam, she said. “Maybe in the future, we would have people dictate reports to more reflect what we do. We’re continuing to work on ways to make the exams more reflective of the actual practice of radiology.”

Some residents are also concerned about the test timing, since interviews for abdominal, neuroradiology and interventional radiology fellowships are a few months before the exam. While it did not affect her, Walker said several in her program interviewed in that time period and submitted rank lists on April 30. They find out the match result the week after the core exam. “Those people have complained that they couldn’t attend staff board reviews because they’re traveling for interviews,” she said, and it has been stressful for them.

Residents are also concerned that they won’t graduate from their residency program as board certified. They’ll be board-eligible and can take the certifying exam 15 months after residency ends. “With the job market as tough as it is, will they hire board residents or hold off until we’re board certified?” said Tahvilian.

“We’re very aware of those concerns,” Vydareny said, noting that the market has been tight for several years independent of the examinations. It’s something the ABR will continue monitoring, especially since the certifying exam hasn’t yet been given. The 2010 class was the first one affected by exam changes, and the initial certifying exam isn’t given until October, 2015.

Any high impact test is stressful, and one with changes, even more so. “We were very anxious and there are probably not any words of comfort I could give,” Bigger said. “As long as people are putting the time in before the exam, most people will find that it was reasonable and that they were well prepared. At the end of the day, as much as any test is a frightening thing to do, I thought it was reasonable. But I’m glad I don’t have to take it again.”