Real Use for AI Tools During COVID-19: Part Two

As the volume of patients infected with COVID-19 continues to rise nationwide, so has the need for rapid patient management capabilities. To answer that call, many artificial intelligence (AI) tools manufacturers have taken steps to pivot their existing tools toward monitoring disease progression and severity.

In Part Two of this video series, Diagnostic Imaging spoke with Eliot Siegel, M.D., professor and vice chair of radiology at the University of Maryland School of Medicine, about the benefits, precautions, and potential long-term impact of using AI imaging tools during the pandemic.

Palmer: What are some of the challenges that we can expect with trying to get tools up and running to the point where we can make these determinations that you've been talking about? What might what might be some stumbling blocks or some difficulties?

Siegel: There are a number of stumbling blocks. And, in many ways, they're really related to many of the long-term stumbling blocks that existed before COVID-19. One of them is being able to gather data. I've had a large number of people who've approached me asking about how can they get access to large databases of cases, most of these databases currently exist outside the United States, because we really have not been using thoracic CT to a large extent.

But, getting permission to use these -- even commercial companies in China have had a really hard time being able to get images from Chinese patients. In fact, at least one of these companies that I know have had to go to each individual hospital and create an individual server to be able to collect and analyze data only at that hospital. They ended up combining the coefficients of the models that they trained at each individual hospital into a larger system. So, even within China, being able to get images outside of an individual hospital for development of algorithms has been difficult.

So, I think one of the big barriers is just getting access to data. And, it's not just imaging data. One needs to have interpretations, and one needs to have the clinical correlative of data. How did the patient do? What were the results of lab studies? What were the symptoms? Being able to collect that information rapidly, especially in the face of so many who are struggling clinically with these searches, has been a major challenge.

Another challenge is deploying these algorithms. How do I get them into my hospital? How do I run them on my PACS system and on my regular workstations? Do I have to have images that I end up sending into a cloud service that has these algorithms and, then, whatever permission issues that I have related to IP permission or business associate agreements? I can tell you at the hospitals where I am working currently, everybody is so focused on issues related to preparation for COVID-19, that getting business associate agreements and being able to get permissions to send images out is really, really difficult.

So, a lot of the barriers become even magnified when there is a surge of COVID-19. So, let's assume that we had a fantastic algorithm to be able to follow a trajectory of disease and quantify disease, look at things that would predict pulmonary function and predict which patients might do better or worse. Being able to get access to those algorithms and send my data out to those algorithms, if they're in the cloud or bringing them inside, is really difficult. One of the things I'd really love to see would be more of an emergence. We're already starting to see this on platforms that run multiple AI applications. Being able to take some of the platforms exist or some of the ones that are being created and being able to relatively easily add an AI algorithm, whether it's FDA-cleared or not, but that could be utilized.

One thing that really made me happy, and it's been reassuring is a large number of academic centers that I've talked with, and also commercial vendors are all suggesting that the software that they're developing can be made, essentially, free of charge. They’re donating a lot of their efforts. I've had a company call me and suggest that they would do annotations for free and adding their software. And, so we have a lot of pieces of these.

The other thing that would be really interesting would be something that has been referred to as ensemble AI applications where one is able to take multiple different companies all doing the same functional algorithm and create multiple together. So, you sort of have a crowdsource of AI applications, or having AI applications that do different things. One that might find the lung fields, one that might look for ground glass, one that might look for lung nodules, one that might look for other things. Being able to combine all of those together -- some of these emerging platforms allow us to potentially do that.

But, at this point, I think mainly, those barriers have kept us from having access universally to the AI software that is being developed and deployed. So, I'm hoping that at least one of the silver linings associated with COVID-19 might be that we learn lessons about rapid development and deployment of AI might be able to get some leverage and might be able to convince some of these hospital systems that being prepared for new AI algorithms that they may need urgently or emergently may thrust them into a position where they are starting to explore. at an accelerated rate, deploying some of these platforms. That should make it much easier to add an algorithm without going through a whole new discussion about servers cloud access, and business associate agreements, etc.

Palmer: So, in that vein, are we thinking about potential long-term benefits? Ironically, when we're talking about COVID-19, benefits from having these barriers broken down? What kind of long-term positives might we be able to see?

Siegel: I think one long-term positive is that there are a lot of people working on algorithms for COVID-19, that might not have even worked in healthcare before. I'll be doing some lectures to computer science folks at a variety of computer science meetings about even what COVID-19 is, what some of the challenges are. They're willing to utilize their own efforts. So, people are getting involved in image analysis that may not have been otherwise. I think we’re learning some lessons about how can one take existing software and, in an urgent situation, be able to train it and pivot it. And, then, validate it also. I think the silver lining is going to be that we've gone through really important rea, drill at this point or real-life drill, unfortunately, where we've had to accelerate deployment and development and experimentation of existing algorithms.

We may have companies working together on this effort that might not have worked together otherwise to create some synergies and being able to take some of these new discoveries that we have and new insights and new collaborations and deploy them in the future for healthcare AI applications. I think is a fantastic silver lining to all this.

Palmer: Wonderful. Well, on the flip side of that, what kind of cautions should our providers take when starting to use these tools and thinking about adapting them or applying them to COVID-19 patients? What do they need to keep in mind?

Siegel: I think it's similar to the ones they need to keep in mind for AI in general. That is, it can be really easy and relatively fast to develop these algorithms. But, actually testing them and validating them can take quite a while. Algorithms that are developed on one machine or one system or on one population may not work on another population. What's really critical is to be able to utilize and deploy these tools with human observers that are looking to see what works and what doesn't work the same way that we who have cars that are self-driving, essentially learn the limitations of the car on different roads.

I think healthcare workers, radiologists in specific, need to learn what are the strengths and what are the weaknesses. With CAD software for mammography for many years, mammographers have learned that it does a really good job with detection and microcalcifications but has certain limitations for finding masses, and I think in a similar way we, as we deploy these AI solutions, have radiologists maintain a level of skepticism as they end up trying the software out and deploying it on in their own environments.

I think, in particular for quantitative tools, it's likely that we'll be able to have better validation. But, for diagnostic tools, it may be a little bit more of a challenge and to have my colleagues understand and realize that a lot of these systems may have been developed for populations that are significantly different than what they have as far as prevalence of disease, as far as clinical symptom and symptomatology and presentation. So, remembering all of those caveats is really going to be critically important and in many ways some of these tools may work like a grammar checker or spell checker. The radiologists will still be doing a fair amount of the assessment, but will be helped by these. And, as time goes on, we'll develop a greater level of comfort with them.

I believe that caveat emptor, essentially, even if the software is free. We’re really important to spend time actually getting a feel for it and validating it, as well.

Palmer: Well, then stepping back to, a 50,000-foot level, when we think about this as a post-COVID-19 scenario, What do you anticipate the impact having been from using AI over the long term? Are re there some takeaway messages that we can get think about in this situation?

Siegel: I think, ultimately, it's hard to predict the future, and it'll be interesting to see the takeaways that we get. I think that we're going to have a much better idea of the relative role of chest radiography and chest CT and imaging, in general. I think what we're going to find is that its main use is going to be in follow-up of patients in the longer term, rather than acute diagnosis. I think we're going to find that its ability to track a trajectory of change is important.

One of the limitations of AI software, as it exists currently, is that it tends to diagnose and to use for a single case a single study. Bu,t in general AI software currently has not really been particularly well developed for tracking trajectory of change over time -- even for mammography software, in general. It doesn't do what most of we radiologists do, which is take a look at how something has changed. The question that we frequently get is, “Is this better or worse, how much better or worse?” AI software in its current incarnation doesn't do that so much. And, I think that much of the software developed for COVID-19, we'll look at trajectory of change, the trajectory of quantification of disease as time goes on, and even quantification of long, more permanent disease and follow-up with that, as well.

So, I think that will be one of the lessons that we end up learning long term. Another lesson will be the impact of being able to pivot with existing software and be able to further develop it, and having faster ways to do that. I think another longer term will be all of the advantages of being able to have multiple different vendors work together and cooperate.

Another advantage, I hope, will be a little bit of a loosening in some of the restrictions that we have on sharing images. I know that we're looking at optimizing patient privacy and security, and that's really important, but having infrastructures that allow us to be able to have ways to have people donate cases, whether they're early countries that are experiencing the disease like China or others and being able to more freely disseminate information to be able to have a central repository, for example, of cases so that we can all learn from each other.

What's been really great has been how quickly journals, such as Radiology, AJR, JRC, etc., have been able to fast-track publications of articles that have shared anecdotal cases, being able to, in an analogous way, create repositories of data and experience both the images records, clinical data, lab data, and putting those together. And, getting those online quickly, I'm hoping will be a longer term consequence where perhaps people will reconsider and make it a little easier and maybe we'll set up some mechanisms in the future to be able to have collaborative capture of those data to be able to be shared and have more researchers be able to develop these algorithms.

Palmer:  Wonderful. Dr. Siegel, I appreciate your taking the time with us today. Thank you so much.

Siegel: Sure. It’s my pleasure. Please stay healthy, and it was great talking with
you as well.

Palmer: Thank you.