Virtual Holography: The Next Step in 3D Imaging?

How do radiologists learn to read and interpret medical images?

Medical images, whether produced by CT, MRI, X-ray or ultrasound, are basically 2D slices of 3D objects. But the process of interpreting those images involves mental spatial calculations that often result in the loss of some clinical information, said Sergio Aguirre, founder and chief technology officer of Echopixel technologies.

Consequently, Aguirre is working on a virtual holographic solution that could improve reading accuracy and efficiency by allowing physicians to see and interact with tissue in space as if it were a real physical object.

Spatial Cognition

To understand the potential clinical value of what Aguirre is calling “True 3D,” it’s important to understand the role spatial cognition plays in the evaluation of medical images.

According to Madeleine Keehner, PhD, a research scientist for the Educational Testing Service and director of the Spatial Cognition Laboratory at the University of Dundee in Scotland, spatial cognition is the ability to mentally represent and manipulate spatial information.

She pointed out that because most medical images show a 2D representation of a 3D object, physicians reading that image need to mentally reconstruct the object. “So spatial cognition involves taking that image and constructing a 3D recreation in your mind,” she said, “and there are big individual differences in how well people can do that.”

Consequently, according to Aguirre, reading and interpreting a medical image is a “cognitively intensive process.”

“Doctors have to mentally deliberate as they evaluate images and they’ll have to try to look at different views to get more information and determine if what they see is the tissue they want to evaluate,” Aguirre said. “A lot of clinically significant information gets lost by looking at 2D views of 3D anatomy.” Aguirre wants to provide is a “true 3D visualization platform” that allows a reader to see subjects as 3D objects that he or she can manipulate and interact with.

Virtual Holography

photo credit: Infinite Z Inc.

Echopixel has partnered with - among others - a company called Infinite Z, which has designed a virtual holographic tool called zSpace. This tool offers a combination of three elements that create an immersive 3D environment: stereoscopy, which provides depth perception; head tracking, which allows the user to look around objects; and prehension, or the ability to actually interact with virtual holographic objects.

“Images appear in open space and appear to do what the brain expects them to do,” said Dave Chavez, Infinite Z’s vice president for hardware engineering. “It allows you to use your intuition and analyze what is going on in front of you.”

A fourth element is necessary in order for this type of advanced visualization tool to be successful in the medical imaging field, and that, according to Aguirre, is the development of an appropriate clinical protocol to take advantage of this open visual space. The recent success of 3D motion picture technology, Aguirre said, demonstrates why an appropriate clinical protocol is necessary to for this kind of 3D visualization to be truly effective.

The movie “Avatar” was widely praised, Aguirre said, not only because it was an excellent movie, but because it was conceived and produced with 3D in mind right from the start.

Consequently, if you compare the quality of the “real” 3D in “Avatar” with other movies in which 3D has been added through a post-conversion process, “Avatar” is going to be far superior. “The 3D experience - and even the movie itself - will suffer in entertainment and visual value [in a post-conversion process] because you aren’t taking into account those additional components and variables you need to keep in mind when telling your story,” Aguirre said.

Having an appropriate clinical protocol in place, then, will be the difference between having “a fun, visual video game and being able to use a professional clinical tool,” Aguirre said. “That’s why we put a lot of emphasis on having that clinical protocol in place.”

So how will this kind of advanced technology affect the relative importance of spatial cognitive ability when it comes to evaluating medical images?

Mary Hegarty, PhD, a professor of psychology at the University of California Santa Barbara and director of the Hegarty Spatial Thinking Lab, suggested that working with holographic 3D images could make spatial cognitive ability less of a factor when it comes to reading medical images. “Spatial ability would probably be less important,” she said. “It’s as if the holographic images are doing the work for the person, so they don’t have to rely on their internal visualization.”

At the same time, she pointed out that research also suggests that in some cases advanced visualization technology is not necessarily a panacea for a lack of spatial cognitive ability. In a study she and Keehner were involved with, subjects were allowed to interact with a 3D object by rotating it on a computer screen. The subjects were supposed to visualize a cross section of that object and then rotate it until they reached that perspective.

“Some people spontaneously did this, while others rotated it, may have got lost and didn’t know where they were,” she said. “And that’s related to spatial ability.

“Sometimes the rich get richer with more advanced visualization,” she said. “But if it is very well defined, or if the tool basically gives you the information you would have needed to visualize it yourself, then I would expect effect of spatial cognition would get smaller.”

And that’s one of the concepts underlying True 3D. The technology allows physicians to grasp, manipulate and look at objects, Aguirre said, and that should affect the role that spatial cognition plays in interpreting images and increase “what we call the intuition part of looking at medical data and we believe this really helps doctors understand anatomy in a much easier and thorough way.”

Virtual Colonoscopy

Aguirre said he believes the potential is there to bring a paradigm shift in the way physicians visualize anatomy, which will “provide a clear clinical benefit to patients.”While the application of the tool could extend to areas like breast tomosynthesis and cardiology, the focus right now is on developing a virtual colonoscopy tool.

“The whole goal of colonography is to screen for polyps, which are the precursor lesions for colorectal cancer,” said Judith Yee, MD, professor and vice chair of radiology and biomedical imaging at the University of California, San Francisco, chief of radiology at the San Francisco Veterans Affairs Medical Center, and a pioneer in the field of CT colonography (CTC). “It’s unlike screening for any other cancer because the goal is to look for the precursor of cancer, and not the cancer itself.”

So a premium is placed on accuracy when it comes to identifying these polyps, because, as Yee pointed out, “if you are able to identify all of the polyps and adenomatous polyps and they are taken out, you can prevent colon cancer from occurring.”

With that clinical goal in mind, “we started with virtual colonoscopy and developed a new navigation strategy based on colon sections,” Aguirre said, adding that it’s a strategy that aims to reduce interpretation times and, more importantly, “minimize a lot of the distortions that are created from certain mappings that some virtual colonoscopy applications have.”

To demonstrate the feasibility of the technology and it potential clinical benefits, Aguirre, Yee and Guillermo Sapiro DSc, of the University of Minnesota, performed a preclinical benchmark test using a public CTC database available from Walter Reed Army Medical Center.

The sensitivity of CTC was calculated based on the findings of an unblinded polyp search using optical colonoscopy as the reference standard. The researchers took cases out of the database from patients who had both a same day optical colonoscopy and virtually colonoscopy and where the virtual colonoscopy procedure had resulted in at least one false negative CTC reading.

“In particular, we were interested in finding those flat lesions that are a different type of polyp and which are twice as likely to be cancerous,” Aguirre said. These flat lesions, called nonpolypoid lesions, are particularly difficult to see.

“We found that we could consistently find those flat lesions compared to the inconsistent performance of other virtual colonoscopy solutions,” Aguirre said. “The reason for that is that we were able to position the colon wall in a way that maximized the subtle elevation difference of the flat lesion from the colon wall.”

According to Aguirre and his colleagues, in the 20 cases processed that included the presence of at least one flat lesion, they were able to increase the detection rate by 20 percent.

“This technology is very new,” said Yee, “and we are still gathering data. But I think it is the next natural step to 3D in that it really brings it to life in front of you and adds an extra dimension of interactivity that we didn’t have before.”

“Having been involved in developing CTC from the beginning, I can tell you that very little impresses me,” she said. “But this impressed me. It’s one of those things there when you see it in real life, it really knocks your socks off.”

A demonstration of the zSpace technology at a laboratory at Olin College in Needham, Mass., illustrated Yee’s point. The technology requires the use of a flat LCD display but the object a reader chooses to study appears to be floating in space. Through the use of a stylus, the reader can click on the object or a section of the object - in this case it was a model of the human heart - grab it and manipulate it, either by rotating it and looking around it, or pushing it aside and grabbing another object.

In the case of virtual colonoscopy, Yee said she is able, through use of a stylus, to take segments of the colon and manipulate them in any way she chooses. “So you can grab these sections, turn them around 360 degrees and look in front of and behind folds for polyps, all in a very quick and time efficient manner.”

And while there is certainly training involved in using the technology effectively, Yee said, “I think readers will find it more intuitive, because it builds on what you have already learned from traditional 3D.”

Aguirre said he believes that this technology has another potential clinical benefit: to level the playing field between large facilities and smaller hospitals and community clinics. “We think that having a platform [like True 3D] enabling doctors to look at 3D anatomy as such, allows them to rely on their medical training and not on their particular spatial cognitive skills,” Aguirre said.

In addition to virtual colonoscopy, other potential applications being explored include breast tomosynthesis, a surgical planning and image guided treatment application in cardiology using coronary CT angiography, and transjugular intrahepatic portosystemic shunt (TIPS) procedures for the liver.

As for the True 3D-CTC application, Yee said she and her colleagues are getting ready to do an initial large study and hopes to have more results available within a year.