Hospital finds tough road to ROE worth the effort

The dramatic rise of informatics in healthcare in the past decade has benefited radiology perhaps more than any other medical specialty. The digitization of image creation, storage, and retrieval has nudged radiology even closer to the core of medical practice.

Yet information systems that optimize device and modality utilization, promote business efficiency, and address patient satisfaction and safety issues remain relatively primitive. Ironically, the rapid growth of high-tech imaging has even provoked a need to conserve resources.

"Information technology vendors are generally unaware of the scope of these needs and how to satisfy them within the context of common radiology and medical practices," said Dr. Daniel I. Rosenthal, associate radiologist-in-chief at Massachusetts General Hospital.

MGH found it necessary to take matters into its own hands by developing a radiology order entry system that does more than merely manage appointments. Their system, generically named radiology order entry, or ROE, also addresses modality utilization, business efficiency, and patient satisfaction issues.

The Web-based ROE solution addresses the latter two matters by allowing any referring clinician inside the MGH Partners firewall to electronically order and schedule in real-time outpatient MRI, CT, ultrasound, mammography, plain-film radiography, nuclear medicine, bone densitometry, and nuclear cardiology exams, along with spine and joint injections.

ROE currently handles 3500 to 4000 examination requests per week, which are ordered by most of the clinical practices across the MGH campus.

"At least 90%, if not more, of our outpatient imaging is now handled by this system, which is what it was designed to do," Rosenthal said.

It was never anticipated that 100% of outpatient exams would be ordered through ROE. Some procedures are not eligible. Interventional procedures, for instance, require direct consultation with a radiologist and are not included. Physicians outside the MGH complex may wish to refer patients but lack access to ROE.

MGH began gradually phasing ROE into the clinical mainstream in 2001.

"Radiology order entry has worked as an efficient and useful program for the busy clinician to order complex or high-cost imaging studies," said Dr. Andrew Freiberg, an associate professor of orthopedic surgery at Harvard Medical School.

DECISION SUPPORT

What makes ROE unique, however, is the recent addition of decision support. This component helps manage device utilization-potentially indicating the direction in which radiology is headed.

The idea of radiology order entry is not unique. Other commercial radiology order entry systems exist, often bundled by vendors as part of RIS packages. But none of these satisfied the demands of MGH. Decision support was high on the hospital's list.

"In our experience, it has been impossible to teach the referring practices to distinguish between known conditions and conditions to be ruled out," Rosenthal said. "Therefore, it is probable that this must be accomplished by the use of examination-specific lists of indications, from which the user may select one or more."

The ROE decision support tool, or ROE-DS, added in 2004 for all CT, MRI, and nuclear cardiology exams, is intended to provide the necessary information so referring clinicians can use radiology exams in the most efficient manner. Business efficiency is a by-product.

"Most radiologists would agree that inappropriate use of imaging exams does occur," Rosenthal said. "Clinicians seem uncertain concerning current indications for imaging."

A 2003 study found that medical house staff are not adequately prepared to choose appropriate imaging examinations for specific indications (Acad Radiol 2003;10(7):781-785). A survey of internal medicine residents revealed that fewer than half scored more than 50% when asked to correctly select the most appropriate diagnostic imaging examination for a series of clinical scenarios. The study suggested that imaging centers consider developing online catalogs to assist clinicians in choosing appropriate imaging tests, the exact function performed by ROE-DS.

Another factor contributing to the rise in inappropriate exams is the practice of defensive medicine, which impels clinicians to order imaging studies even though the possibility of significant findings is remote.

"Imaging is increasingly being used as an alternative to physical examinations, especially in emergency settings, a practice that inevitably results in low diagnostic yield," Rosenthal said.

He added that patient expectations also drive imaging demands. Although not necessarily contributing to medical management, a negative examination often helps reassure patients.

WHAT'S THE SCORE?

MGH designed ROE to be a simple, user-friendly tool capable of capturing all information necessary for decision support, scheduling, examination performance, and ICD-9 coding.

"We have replaced the individual transaction approach with a physician scorecard that reflects each physician's overall performance in ordering examinations," Rosenthal said.

Rather than a simple binary go/no-go approach, ROE-DS simultaneously provides comparative scores for various examinations that might be ordered in a particular clinical context, thus serving as both an educational aid and a patient management tool. With healthcare insurers increasingly concerned about the rising utilization of high-cost imaging, targets are being built into reimbursement schedules.

Rosenthal said ROE-DS scores are meant to accomplish three things: help limit proliferation of low-utility examinations, justify the appropriateness of exams that are ordered, and skirt cumbersome healthcare insurer gatekeepers. A spinal examination, for example, would receive a low utility score if the request indicated only back pain, but it would be considered appropriate if the physician also provided information that the patient had abnormal extremity sensations.

ROE offers indications for each exam as a series of check boxes, grouped in three categories: signs and symptoms, known diagnoses, and abnormal previous tests. At least one item must be checked. Suspected diagnoses or rule-outs alone are not sufficient for ordering.

Once users indicate the order is complete, ROE-DS provides instantaneous decision support obtained from a lookup table that lists a numerical utility score for the test being ordered. ROE with symptoms, not disease conditions, currently numbers approximately 10,000 reason-test pairs. Utility scores are derived from published sources, particularly the American College of Radiology appropriateness criteria. ROE-DS utility scores range from 1 (very low) to 9 (very high).

A high score (7 to 9) indicates that the information provided strongly supports the use of the imaging test requested. A low score (1 to 3) indicates that the examination may not be appropriate. In this case, other choices commonly used to evaluate the condition will usually be listed, with their utility scores shown. If a clinician is placing an order for a CT of the head for a specific indication, for example, ROE provides a score for head CT and an alternative score for head MRI.

"By anticipating imaging alternatives, the system allows referring clinicians to compare exam scores to see which one may be more appropriate," Rosenthal said.

A low decision support number does not prevent the clinician from ordering the examination anyway. But utility scores for each physician and each examination type are collected over time, and reports are scanned regularly for scores that lie consistently beyond the margin of convention. For very low scores, the system provides the physician with the telephone number to call MGH radiology to discuss the rationale for the examination. The radiologist will not refuse the examination but may suggest alternatives.

SPEED BUMP

Dr. Lynne Brodsky, a primary-care physician in the MGH Chelsea Healthcare Center, said use of ROE leads to more efficiency.

"Standardization means that ordering will better adhere to applicable clinical guidelines," she said.

Unlike other utilization management systems, the scores generated by ROE are not meant to be roadblocks, Rosenthal said. ROE doesn't quibble over details by trying to determine if a specific exam is appropriate or not, as most existing utilization management systems do.

"Gatekeeping is not a very efficient way to manage utilization because medicine consists of a million special cases," Rosenthal said.

Gatekeepers also fail to distinguish between clinicians who frequently place inappropriate orders and those who rarely or never do so.

Rosenthal considers it far better to look for individual doctors who consistently order exams judged to be of poor utility. He added that increasingly common pay-for-performance programs, which reward providers financially for achieving contractual goals, often put radiologists in the role of gatekeeper.

"This is not the best approach, since it may strain the consulting relationship and result in conflict of interest for the radiologist," he said.

ROE-DS not only helps clarify imaging appropriateness, but it may also tend to decrease the number of requests. Initial clinical experience with ROE-DS has shown that studies were canceled following receipt of a low utility score 19.8% of the time. In most cases, however, the user proceeded to schedule the exam even after a low score. The two most popular reasons cited were "recommended by specialist" (55%) and "disagree with guidelines" (25%).

"It is clear our system has had an impact on the way in which imaging studies are ordered, as shown by the declining number of low-utility examinations," Rosenthal said. "It is less clear whether this represents a change in utilization practices."

Some clinicians persist in ordering studies for screening when there are no indications.

"Since ROE requires that an indication be specified, retention of the conventional ordering and scheduling system allows such cases to be handled on an individual basis," he said.

ROE-DS may not be a roadblock, but it does serve as a speed bump.

"When you are prompted as to whether it is an appropriate test, you tend to think twice," said Dr. Eric Weil, medical director of the MGH Care Management Program. "I suspect ROE has cut the overall request rate."

Following the first year of clinical experience with decision support, Rosenthal found that the system was much more successful with primary-care physicians than with specialists. Specifically, primary-care physicians increased their log-in rate from 33% to 73%, while specialists tended to continue to use support staff for placing orders. Neurology, for instance, increased from 5% to 12%, and orthopedics from 2% to 5%.

Rosenthal said that specialists interact differently with the system. They tend to write down orders for the same few exams and give them to their secretary.

"Providing decision support to a secretary is useless," he said. "Primary-care people want to see whether their score is good or bad. Specialists don't really care."

Freiberg said ROE-DS has not changed the way in which he orders MRI or CT exams.

"As an orthopedic surgeon, I always want to view and interpret plain x-rays before obtaining these studies," he said. "Also, we have the advantage of understanding the functional anatomy and performing a history and physical exam prior to ordering high-cost studies."

IMMEDIATE APPOINTMENT

One important ROE feature that promoted rapid clinical acceptance was the immediate availability of appointments. Office staff no longer must call the radiology department to schedule appointments. They can schedule exams while the patient is still in the office, which is more convenient for the patient as well.

"The time required to interact with the computer is somewhat greater than would be required to complete a paper request, but the time saved in scheduling the appointment more than compensates for this," Rosenthal said.

Use of ROE and ROE-DS by clinicians is voluntary, in much the same sense that balancing a checkbook is voluntary. Due to an agreement with several insurance carriers, use of ROE-DS replaces the need for telephone preauthorization, and therefore it is strongly encouraged.

"ROE is pretty much the system of choice across all of primary care, but it's not really voluntary," Weil said.

As with other medical informatics systems, ROE eliminates paper from the process, so one advantage of the system is that radiology requests never get lost.

"I can easily schedule appointments myself if I happen to be talking to a patient off-hours or in a nursing home. Or I can enter the clinical request, and my secretary can make the appointment," Brodsky said.

Booking radiology exams in real-time appeals to both clinicians and patients.

"ROE has definitely improved practice life," Weil said. "It's time-saving for the patient to receive all necessary exam information at the exact moment of scheduling, and there are typically no questions from the radiology technician or radiologist when the patient arrives for the exam because the prompts make everything clear."

Rosenthal said a radiology order entry system must replace, not supplement, prior methods of ordering.

"A telephone call should not be necessary to verify an electronic order," he said.

NOT PERFECT

The system has room for improvement, however. Some clinicians believe ROE has a tendency to run slowly, which can be frustrating for someone accustomed to instantaneous computer response. The biggest issue is that some known indications for some tests are not included in the available menu.

"The ordering clinician sometimes has to 'lie' and select an untrue indication, and then type the actual indication in free text to get the study done," Brodsky said.

Current guidelines, for example, recommend screening for abdominal aortic aneurysm in men over 65 who have ever smoked, but ROE does not have that listing on the menu for ultrasound, which is the preferred screening modality, she said.

There are ongoing efforts to manage and adapt the system, according to Rosenthal. In 2005, for example, the American College of Cardiology came up with an entire new set of guidelines for nuclear imaging of the heart.

"Whenever something like that happens, we have to look at what the ACC recommended, determine whether it is the same or different from what we recommend, and then decide which one is preferable," Rosenthal said.

MGH's experience with ROE has been so encouraging that commercialization is imminent, Rosenthal said. ROE has been licensed to Commissure (New York, NY), which is writing a commercial version of the system that is expected to be available later in 2007.

"The order entry process has an impact upon many aspects of healthcare delivery and offers many opportunities to introduce tools that can enhance the use of medical imaging services," Rosenthal said.

Mr. Page is a contributing editor to Diagnostic Imaging.