Q&A: Improving Radiology through Data Search and Mining

Woojin Kim, MD, wanted the same functionality as Google for quickly finding and reviewing past radiology cases. So he built one. Kim, assistant professor of imaging informatics and assistant professor of radiology at the Hospital of the University of Pennsylvania, is the principal architect of Montage software, which searches radiology and other clinical reports.

But Kim went beyond simple search, adding capabilities to analyze the data and put it to use in practice. Diagnostic Imaging spoke with Kim about the evolution of his solution and the future of data mining in radiology. The following is part of that discussion. Kim is also scheduled to speak during a session on advanced enterprise work flow at SIIM 2012 next month.

What did you want to do and what problems did you see that prompted you to develop the search and data mining tool Montage?

Around 2009, as soon as I became a full-time faculty at the University of Pennsylvania Hospital, I wanted to be able to search for studies I had seen in the past. If I were to say I saw a particular case, a tendon tear in an MRI of the shoulder and wanted to look for that particular case, unless I remember the patient’s name, I had no easy way of looking up that study. When I am looking for a case, I kind of know what I said in the reports, and the rough time period. Most doctors have that kind of memory, but many of us don’t pay attention to patient’s name when we dictate.

Initially we had some rudimentary search application, but it would take a long time and it was not very user friendly. We go online all the time and use search engines like Google to look up pretty much anything online. If Google hangs for three seconds, you would think there’s something wrong with your ISP. We are spoiled; the response has to be immediate. That kind of functionality is demanded by the current users, so I wanted to build an application where I could easily search radiology reports and cases I have done in the past by remembering certain elements – not patient demographics but description of findings, for example, and get those results in less than a second.

That’s what prompted me to design the very first application at Penn, called PRESTO, which stands for Pathology-Radiology Enterprise Search Tool. We named it that way because we wanted to take it one step further. What if we can search another specialty in medicine like pathology and do the same thing with pathology reports? Even taking it further, what if I can - with a single click - ask my search engine to show me all past path-proven cases of prostate carcinoma with a certain Gleason score where the patients also had MRI of the prostate at my institution?

For that search to work, the search engine has to first go into the pathology database and find all past path-proven cases of prostate cancer and then go into the radiology database and find the prostate cancer patients who also had imaging studies. Finding that intersection between the two databases - I wanted to be able to do that in a second.

Why is that so important, to access that past information, and how useful is it to have that information?

In the days before PACS, radiologists would have regular interactions with other clinicians. During that interaction, the surgeon would say. ‘Hey remember that case, it was a great call. It ended up being exactly what you said.’ Or, ‘That thing you called cancer, we went in and excised it and it turned out to be not malignant.’ We used to be able to get this kind of feedback.

Whenever I see a lesion, I describe it, give differentials and give my best guestimate as to what that may be. But unless you’re so diligent in checking the EMR for pathology reports, you won’t get to find out if the call you made was right or wrong.

But with PACS and EMRs, and the clinicians are very busy, they don’t have time to walk down and have discussions of cases. Radiologists now are focused on cranking out the cases, and they are getting little to no feedback. We are training a generation of radiologists who don’t have a whole lot of interaction with the clinicians, and they aren’t getting the feedback they need to improve their reading accuracy.

That’s the idea. This allows you to teach yourself and to stop calling a particular finding malignant every time you see it when it’s really benign.

Does this database include images or is it written reports?

It just comes up with the written reports. But what happened was I would go and talk about this particular search engine and people would get very interested. After each presentation, people would come up and ask for me to install it in their institution. So we decided to commercialize this, so that we have a venue to install this outside of Penn.

That’s how Montage got formed in 2010 with the goal of being able to distribute these applications. Montage can import radiology reports on a daily basis and do the same with pathology reports (or any other free-text reports from various different sources). If Montage today gets 50 pathology reports, it would take those reports and go back six months - or whatever number of months you’d like to have it go back - and look at every radiologist who ever encountered that patient, do a smart match with that pathology report and past radiology reports.

The automated follow up module will report a match, so if there was a sarcoma of the foot that was resected and a particular radiologist read this person’s foot MRI prior to pathology confirmation, the module will notify the radiologist automatically because he might be interested in the pathology report. The interface is similar to an inbox in Outlook. So you get all the cases of pathology that might be related to something you read in the past six months, for example, without having to do anything different.

There are a few design concepts that I’ve put significant importance in. One, the speed is very important for reasons I’ve described about the usage of Google. Second is simplicity. Initially when you use Montage, you have a single box to interact with. We wanted the application to be intuitive and simple to use without having the need to read a 50-page manual before using it. Have you ever read a manual for Google? Remember when you were first introduced to Google? You go to the site and you enter your search words. Simple; and that’s the kind of experience I want our users to have.

That covers the basic search aspect of the application. But there are a lot of metadata associated with each radiology report. All that information can be mined to create business intelligence and analytics tools. Montage takes the metadata and creates these analytics tools. Say you are a practice president or chairman of a radiology department, you can get a global sense of what your department is doing. But you can also drill down to a single radiologist level to get a personalized score card of her performance.

Some of the bread and butter metrics in radiology departments include volume of studies as well as turnaround times. We can use those metadata to generate the business intelligence tools giving you deeper look into those metrics. If you have ever used Kayak to make travel reservations, you will notice that it gives you a lot of options without overly complicating the user interface, and we designed the application just like that. The whole user experience is similar to making an airline reservation online.

We’ve been generating other kinds of business intelligence tools as well. With decreasing reimbursement and people not retiring and increased competition among practices, people want to know their referral patterns. Are we losing referrals from somewhere? Can we gain it somewhere?

If you look at referral patterns, it will show you your top referring physicians, but we decided to take it one step further. Let’s look at where the patients are actually coming from, and we put information on a map to show you like Google maps. This way, you can easily see where your patients are coming from in relation to your hospital or imaging center. Then you can ask yourself, I see big gaps here and there, is that a potential area of growth for us?

We paid a lot of attention to data visualization, because you can slice the data in many ways but depending on how you visualize will impact how the end users analyze it and maximize the use of the data.

What is unique about Montage analytics is the quality control module. We first started off by being a search engine and focused on analyzing free-text. We started as a search company. We try to understand text. Leveraging on that, we were able to create a quality control dashboard, for example, with critical tests result (CTR) analysis. How many CTR cases do you have and how do you keep track of the frequency of reporting the critical tests results? Because Montage can look at the reports and analyze them, it can look at all the incoming reports and determine which report contains any of the CTR elements. Then it will flag them accordingly.

Why was it important to have all these components in a single system? It seems like you could have had just the search engine and that would have been so useful alone?

When we show the search component, the radiologists love it. But the administrators are more interested in business analytics and quality control. They are more interested in how many cases of X they are seeing and how are we doing in terms of turnaround time. Well, it’s all part of the same dataset. Different people want to look at different parts of it. We wanted to build an application that would satisfy these different needs.

Will there be a point where these data mining tools become so ubiquitous that you won’t be able to imagine a world without the access to this data?

Yes. If you look at the national trends with meaningful use and people paying attention to pay for performance, the interest is in quality. The only way to meet those requirements is first to measure it. You really need access to that data. So I think people will demand it and if they don’t the government mandates will force people to look into it. When they get used to it, they will say, ‘Wow this is great stuff. You can do lots of things.’ And then institutions will want to share the data. The government might want to do it on a state or national level, and I think that’s when we can have some serious impact on national health care and bring down cost as well.

As we talk about data mining, what do you expect over the next 5 to 10 years? Will be it larger institutions and the government tapping into that information and then expanding on it?

I hope so, but it probably won’t happen in five years. It’s a slow process, but once you reach a tipping point, there will be a flood of metrics and flood of capabilities. The ability to combine radiology and pathology is just the tip of the iceberg, for example.