Improving Adherence to Best Practices for Incidental Abdominal Aortic Aneurysms on CT and MRI

Could an emerging artificial intelligence (AI)-powered tool improve the consistency of best practices in radiology reporting for incidental abdominal aortic aneurysms found on computed tomography (CT) and magnetic resonance imaging (MRI) scans?

In a recent study, published in the Journal of Clinical Medicine and Experimental Imaging, researchers found the use of an AI prompt during radiology reporting of incidental abdominal aortic aneurysms led to a greater than 51 percent improvement in adherence to best practice recommendations (BPR) for reporting of these incidental findings.

During recent interviews, study co-author Eric Rohren, M.D., a professor and chair of radiology at the Baylor College of Medicine and associate chief medical officer (CMO) with Radiology Partners, and Krishna Nallamshetty, M.D., a professor and chair of radiology at the University of South Florida and the CMO for Radiology Partners, discussed the development of the BPR, the AI technology and the potential impact for reporting of incidental abdominal aortic aneurysms.

What are some of the challenges with managing incidental findings of abdominal aortic aneurysms?

Dr. Rohren: One of the major challenges is (abdominal aortic aneurysm) is not an infrequent finding on abdominal CT (computed tomography) scans or abdominal MRI (magnetic resonance imaging) scans done for other reasons. It's a manifestation of vascular disease, and it's seen when the abdominal aorta starts to dilate in response to atherosclerotic disease. As radiologists, because it is seen on such a frequent basis, we tend to get a little bit blase about reporting abdominal aortic aneurysms. We think, well, it's just a small abdominal aortic aneurysm (with) no immediate danger to the patient. You might mention it briefly in the report, or you might not even mention it if it's just a little bit dilated.

But the problem is, we know that these abdominal aortic aneurysms tend to dilate over time. You may see the patient at one point in time (and he or she) may have an aneurysm. Normally, the abdominal aorta is up to about 2.5 to three centimeters in size. You might see one that is 3.5 or 3.8 centimeters so it is little bit, somewhat dilated, but over time that will progressively enlarge. That's not a reversible condition. That patient, in the absence of any specific recommendations for follow up, you know, may go about their life and may not have an opportunity for another scan and … that abdominal aortic aneurysm is dilating until it gets to a point where there is significant risk. Unfortunately, the end result of abdominal aneurysms can be catastrophic. The end result is like a balloon. They dilate to the point that they become very fragile and eventually will rupture. … When it ruptures, it's almost always a fatal condition. Patients present to the emergency room with abdominal pain in severe distress, emergency surgery has to be done at that point to try to save the patient's life, but the outcome is usually not very good. The survival of those patients is quite poor.

So we have opportunities as radiologists seeing these patients early on to identify the aneurysm before it has a chance to get to that point and get them on a pathway for either imaging surveillance to document how quickly that abdominal aorta is dilating, or, in some cases, if it's even larger, for specific intervention to treat the aneurysm before it gets a chance to rupture. There are endovascular approaches to put in stents to be able to stop the progression of the aneurysm and prevent that catastrophic rupture down the road.

I think that's the major challenge. If you look at radiology practice at a national level, it's getting all of the radiologists, thousands of radiologists, to not only recognize these abdominal aortic aneurysms, but to consistently message the appropriate care for that patient.

Dr. Nallamshetty: Usually, the guidelines require that the patient requires some additional follow up that could be imaging-related, or it could be a consultation with a vascular specialist, either an interventional radiologist or a vascular surgeon. But we've seen a lot of variability in what radiologists recommend. That's where I think the opportunity is. How do we streamline what we recommend for follow-up based on evidence-based guidelines?

Were there any other factors that sparked the initiative to develop these best practice guidelines?

Dr. Nallamshetty: Absolutely. Very early on, we were starting to look at ways that we could improve quality in radiology as a specialty. We started by asking a combination of radiologists as well as hospital partners what defines good quality in radiology. The most consistent answer was turnaround time, which basically refers to how quickly we read the studies. If you think about that, first of all, that's not really a quality metric, right? That's more of an operational metric in terms of how quickly you read the study, but I would even argue, the faster you read the study, probably the less quality you're going to have because you're speeding through it.

We thought about a more meaningful way that we can measure quality in radiology. If you look outside our specialty, evidence-based guidelines are key in modern medical practice, right? Regardless of specialty, there are best practices and evidence-based guidelines. If you adhere to evidence-based guidelines, there are better patient outcomes, decreased cost of care, and consistency in treatment.

We looked at an opportunity for evidence-based guidelines that exist in radiology, and we partnered with a few academic medical centers early on, including Johns Hopkins University, in looking at thyroid nodules. That was the very first one that we looked at. Interestingly enough, there were clear evidence-based guidelines for how you should manage a thyroid nodule but when we did a brief like evaluation of how often that is actually done, we were shocked because the adherence rate to these evidence-based guidelines was anywhere from four to 30 percent of the time. When you look at radiology as a specialty right now, to me, if you think about that extreme example of four percent, we're recommending the right thing four percent of the time … . It was just a huge opportunity for us to really make an impact on patient outcomes. That was really the birth of how we take evidence-based guidelines, make those mainstream in our practice, and then really give our radiologists data and tools to be able to adhere to those guidelines.

Can you talk about the improvement you saw in adherence to these best practice recommendations over a two-year period?

Dr. Nallamshetty: When we first started (looking at adherence to evidence-based guidelines), whether it was our practice academic medical centers or other private practice groups, performance was about the same, and I would say it was honestly mediocre.

The reason I think that adherence was so low is because there is a separate guideline for each finding that we see in radiology, right? In radiology, there are tons of different findings that we're looking for that we pick up incidentally. Each one has its own set of guidelines, and each of those guidelines usually change every year or two years, so it's really difficult for physicians to keep track of all the latest advancements in every single guideline.

We started looking at AI solutions very early on in like the 2014 and 2015 time period. We built our own tool called recoMD and what it does is it listens to radiologists as we report our exams. … Every time I mention something for which there is an evidence-based guideline, I get a little pop up on my screen that says, ‘Hey, Doc, you mentioned an abdominal aortic aneurysm. It's four centimeters. Based on the current guidelines right now, here is the recommendation that you should be suggesting.’ I look at that, and as long as it has captured that correctly, I can hit a little thumbs up button and it auto inserts it into my report.

As the radiologist, I don't have to manually keep track of every guideline update that comes out. The program does it for me, and as long as it's captured correctly and I hit the thumbs up, it's consistent, right? If my specialty is brain, and your specialty is abdomen, and there's some other finding that comes up, we may not be up to date on the most recent guidelines, but here the recoMD becomes an educational tool, because it's showing me in real time what the guidelines are, and then I'm consistently inserting it in.

Dr. Rohren: The abdominal aortic aneurysm best practice recommendations (BPR) are one of the earliest ones that Radiology Partners worked on because of the importance for patient care. We have data going back many, many years, looking at practices before the deployment of this program and after the deployment of the program. Like we've been talking about, the success of the program is getting all of the radiologists to report in a consistent fashion and make those recommendations. (Initially), we found that that compliance with the BPR was rather poor at under 10 percent. That's not to say that the radiologists were doing a bad job or missing things. They may have commented on the abdominal aneurysm, but just weren’t making any specific recommendations for what the best care for that patient would be.

Once we implemented the program, and after a short period of education and feedback, we found that those numbers rose quickly upwards of, I think, 80 percent or higher of reports being compliant with our best practices recommendation. We were able to see that as a group, the radiologists were all of a sudden reporting consistently and making the best recommendations for the patients when they saw abdominal aneurysms of all sizes. The BPR breaks it down. What do you do with the small aneurysm? What do you do with the medium-sized aneurysm? What do you do with the large aneurysm? Like I said, it may be imaging surveillance, it may be referral to vascular surgery, it may be a recommendation for intervention, even surgery or an endovascular approach. (However), giving that specific recommendation we feel is very important in our role as radiologists.

Dr. Nallamshetty: One of the things we do is we take every site before we implement these tools and look at what their baseline performance is. Then we track it monthly at an individual-like practice level, locally, and even down to the individual radiologists. Every month, people get their individual performance so they can see how they're doing. There are also opportunities for learning as part of our peer learning program. If they mentioned something and it was not a best practice, there's an opportunity for them to click in and then learn more as to why that there was a discrepancy.

We've shown consistently across all of our best practice recommendations that the improvement in the adherence rates with our technology that I'm talking about has been anywhere from two to five times that of the baseline performance. We have seen a 50 percent (adherence) improvement in some (areas) but we've seen significantly more in others across the board, and that's, I think, the combination of education as well as technology and AI that really unlocked the potential for us to perform even better and do better quality work for our patients.

Dr. Rohren: At a practice level, we provide a score sheet for the practices showing, how often you're compliant with these various BPRs. … Each practice will get a scorecard showing you know, you're in red, you're below where we'd like you to be, you’re in green, you're at or above our target for the compliance. Those reports are delivered to the practice leadership, and then they share them with the radiologists in their practice. We don't try to make this a punitive process. We try to make it an educational opportunity. We share those score sheets in an anonymous fashion with practices around the country to hopefully institute a little bit of competition amongst practices to see who can perform the best. It really comes down to that feedback, constant feedback to show okay you're doing well, you're where we need you to be. Bu then a month or two down the road, maybe you're starting to drift a little bit and it's time to have a little bit of a refresher. The educational program is always available in some settings. It's integrated into the radiology reporting system for the Radiology Partners practices.

When somebody says there is an abdominal aneurysm, there will actually be a pop up in the dictation system that says, ‘Would you like to use these particular recommendations depending on the size?’ Not all of the practices nationwide have that degree of IT integration, and so sometimes it's a little bit more of a manual process. However, there are online resources at the Radiology Partners website that outline all the BPR recommendations so it's very formulaic in terms of how radiologists can follow the guidelines.

How does the BPR for abdominal aortic aneurysms differ or compare with other recommendations out there?

Dr. Rohren: These guidelines really take into account the recommendations that are already out there. These were not developed totally in house based on internal opinions. We lean heavily on recommendations that are out there. The (abdominal aortic aneurysm recommendations) that we use are based very strongly on the Society of Vascular Surgery practice guidelines. The vascular surgeons are the ones that ultimately end up dealing with these from a treatment standpoint, either through surgery or endovascular repair. … Of course, we take a look at the American College of Radiology (ACR) and the Society of Vascular Surgery and Society of Interventional Radiology guidelines.

We try to ingest as many of these recommendations as possible, and then we have what we call the national subspecialty group led by a leader in that field within Radiology Partners review these recommendations. Each of these groups may be comprised of abdominal imagers that would be overseeing this particular practice guideline. As a group, they review the various guidelines that are out there and develop the internal guidelines that Radiology Partners is going to use. Our guidelines are internally developed but based in large part on recommendations from other specialty societies.

Dr. Nallamshetty: There are some differences in the published best practices and what may be implemented practically. I'll give you an example. Sometimes when the best practices come out, there are guidelines on how to follow up certain things, but there are some gray zones. It may not have captured a certain scenario, or if a patient has an underlying problem, you might not know exactly where to fit them in into the guidelines. We have put together a team of what we what we call our national subspecialty boards that have key thought leaders from all over the country in every subspecialty in radiology. Then what we do is we take that guideline to that group, and different scenarios are posed Well, what happens in this instance when the patient has X, Y and Z, and that group will then work with outside academic medical centers and universities to kind of think about and debate what's the right thing to do in those gray zone areas, and we also incorporate that into the guidelines.

That way, when we actually launch in the technology platform, the recoMD captures not only what exists there, but it's even more detailed because now we've addressed some of those gray zones that are missed in the published guidelines.

Is there anything in the BPR for abdominal aortic aneurysms that may surprise radiologists or things they may not have been aware of?

Dr. Nallamshetty: It’s a really interesting question. Typically, in radiology, we practice our niche or our subspecialty, so I might know all of the nuances and the latest when it comes to my specialty, but I’m not up to date on everything that might be happening in another area within radiology, right? What we see typically is people practice the way they were trained during their residency programs, which might be a long time ago. An interesting side effect of this program is it became a way to teach radiologists in real time as to what the guidelines are. Sometimes I'll get feedback from our physicians like, ‘Oh, I didn't realize that they changed the guideline, and now an aneurysm that measures this centimeter requires this type of follow up, because when I was trained, it was done differently.’

I think that's kind of the beauty of what we've built. It's not only an educational platform, which, to your point, surprises the radiologists when it's something different than what they were trained on but it also helps make a meaningful impact in following the guidelines.

Dr. Rohren: I think it's hard for us as radiologists to be as clearly spoken as adherence to the guidelines would require. You know, we tend to be a little bit observational in how we perform as radiologists. You know, my job is to see and describe things, but my job is not really to tell the referring clinicians what to do. This is a little bit of a sea change as radiologists sometimes have a hard time making that adaptation to realize that sometimes we are the experts that not only see things but have the ability to make recommendations for the best follow up with their patients. It may be hard for us to say, I see an aneurysm of this size, and I recommend this patient go see vascular surgery, for consideration of surgery or consideration of endovascular repair. I think a lot of radiologists are uncomfortable and would like to just be in the background and say, I see this, but not make any specific recommendations.

But as I said (earlier), the downside of that is if we don't emphasize the importance of the particular finding and make a specific recommendation for how that finding should be dealt with by the person who ordered the study, that patient may end up falling through the cracks, not get the appropriate follow up, and then be at risk for that catastrophic outcome. You know, we're talking about rupture in the case of an abdominal aortic aneurysm. … Making these recommendations will hopefully get these patients on the appropriate pathway toward the best diagnosis. Part of it is (facilitating) the best medical outcome and part of it is utilization of limited health-care resources, not over imaging patients who don't need a follow up and making the best recommendation.

For Related Content

• “New AI Algorithm for Abdominal Aortic Aneurysm Detection on CTA Gets FDA Nod”

• “Novel PET Tracer Capable of Predicting Abdominal Aortic Aneurysm Rupture”

• “Abdominal CT Study Shows 20 Percent Reduction in Iodine Contrast with Photon Counting CT”

These guidelines are a tool for radiologists to become more actively involved in patient care and to take on the responsibility for making those recommendations that are ultimately going to lead to the patient receiving (appropriate) care.

Is there anything else you would like to add about the best practice recommendations?

Dr. Rohren: This is something that we as a radiology community at large need to be looking at. As I said, health-care resources are limited. (With) our aging baby boom population, the need for imaging is going up. There is a national shortage of radiologists right now so we really want to make sure that we're being as impactful as possible and doing appropriate imaging in a way that leads to the best value for the care that we do provide.

Radiology Partners is focusing on this with the BPR program, giving their radiologists the tools to appropriately manage all of these incidental findings in a way that is best for the patient and preserving our precious workforce. I think there are lessons to be learned at a national level that we as radiologists, in all settings, academic, private practice, across the board, should be considering these sorts of approaches. I know the ACR is also very involved in developing recommendations, so coming up with some sort of national consensus that we as radiologists can all get behind is ultimately going to be very important for our specialty at large.

Dr. Nallamshetty: Our (BPR) program is constantly evolving. When we started the program, we started with thyroid nodules, like I mentioned. Since then, we have added additional pathologies that we would like to track, and we selected those because we thought they were the most impactful for patients. Two, we felt like there was a ton of unnecessary medical imaging being performed. Thyroid is a good one in the sense that there is a lot of over utilization of imaging, procedures, thyroid biopsies, etc. We thought that's a really good opportunity to help our patients by decreasing unnecessary work, which, from a patient perspective, creates anxiety and all of those other factors, and really focusing on the guidelines that we know will have an impact on outcomes. Currently, we have six guidelines that we are actively tracking, promoting, and measuring for our radiologists. When we start performing at a level that we want to, and we've set a really high threshold, we can continue to add on other best practice recommendations with time.