How the Right Operations Data Leads to the Wrong Analysis

I recently attended a great session put on by the AHRA on the future of economics in San Diego, Calif., as well as several sessions at the RBMA’s Fall Education Conference in Arizona. The underlying themes were that with financial constraints on the health system, everyone will have to do more with less. Best of breed data analytics will enable us to do more with less, without comprising quality.

We need real-time metrics that provide us with actionable information. We need to marry clinical and financial data. We need to push data to radiologists, technicians, marketers, and billing. We need to use data like other industries are using them. Fast food restaurants, retail, and professional sports all use data better than health care. Some industries live or die by their use of data, such as Internet based companies, financial services and insurance companies.

There is a quality in the tools and in the analysis. We accept that one radiologist can be better than another at interpreting images, even after identical and lengthy schooling and experience. All analytic tools are not equal and all analysts are not equal.

Just because you have reporting or dashboards does not mean that you have a quality tool. Perhaps the most important measurement of a tool is based on what kind of data is available to the tool. Are you analyzing clinical or financial data? Are you able to view various fields, like cash collected by referring physician and turnaround time? Can you visualize the data to tell a story? Can you distribute the reports to the right people? Is the data timely? Can someone manipulate the data before distribution? Or worse, is someone forced to manipulate the data in order to distribute?

Deciding which reports to create and how to interpret the results is a more subtle skill. To demonstrate this, let me show you how to lie using statistics. I can show you how to factually represent underlying data - while simultaneously distorting the meaning. The sample is based on Ascombe’s quartet, four datasets that are identical statistically, but appear very different when viewed graphically.

It is important that they are identical statistically, because this is how you could predict what the next point in the series would be. The trend in the data would suggest a new point along the blue line, even though looking at the existing points, any new point would probably not fall on the line.

Let me give you a quick overview of the chart below. The dots below represent an occurrence of some activity. Let’s assume the x axis (on the bottom) represent months. The y axis (on the left) represents the number of referrals from one physician, by modality during those months. You can see from the trend that it is up and to the right. This is the trend that of course you would like to see from all your referring physicians.

Set 1 - There is some unexplained variation, but with a trend up and to the right. You could see this type of pattern on a month-to-month basis.

How to misrepresent the data: If you are presenting for a high performing month highlight the increase on a month-over-month basis. Ignore the overall trends and focus on the unexplained variance - but create a narrative to support - such as the dinner or golf outing. (For marketers make sure to take credit for months when there is an increase.) If you are presenting for a down month make sure to only show the increase over the first period and year-to date totals, this will hide the unexplained drops.

Set 2 - Here we see a pattern of rising referrals that peak and drop off.

How to misrepresent the data: Make sure that you ignore the drop off and focus on the percentage increase from the beginning of the year. Maybe even group each month into a quarter or most recent three months. The main goal here is to mask the downward trend.

Set 3 – This shows very consistent increasing referrals, with large variance seen in the one month, only to revert back to previous trend.

How to misrepresent the data: Use the aberration to project higher referrals in the future. When an outlier like this occurs, take credit for the increase. Shine a large spotlight on it. Hide the decrease in the following month by just showing year-to-date amounts.

Set 4 - In this set, there is nothing in the beginning, a lot of various modalities referred in one month, nothing for several months, and then a high level of referrals months later.

How to misrepresent the data: You can use all the same tricks as from Sets 2 and 3. The key, however, with this data is to act like it doesn’t exist at all. Lump it in to “other” or exclude it as an outlier.

Doing more with less requires doing better with what you already have. You have a lot of data. Hidden inside the data are the efficiencies that you need for better financial margins. Make sure that you are getting the right data, with the right tools and right analysis.

Graph credit: Wikipedia

David Fuhriman is CEO of Bern Medical, a data specialist company. Bern provides dashboard reporting to iPad/iPhone, billing audits, and other data analysis services.