The virtual mirror: a new interaction paradigm for augmented reality environments.

Medical augmented reality (AR) has been widely discussed within the medical imaging as well as computer aided surgery communities. Different systems for exemplary medical applications have been proposed. Some of them produced promising results. One major issue still hindering AR technology to be regularly used in medical applications is the interaction between physician and the superimposed 3-D virtual data. Classical interaction paradigms, for instance with keyboard and mouse, to interact with visualized medical 3-D imaging data are not adequate for an AR environment. This paper introduces the concept of a tangible/controllable Virtual Mirror for medical AR applications. This concept intuitively augments the direct view of the surgeon with all desired views on volumetric medical imaging data registered with the operation site without moving around the operating table or displacing the patient. We selected two medical procedures to demonstrate and evaluate the potentials of the Virtual Mirror for the surgical workflow. Results confirm the intuitiveness of this new paradigm and its perceptive advantages for AR-based computer aided interventions.

Virtually extended surgical drilling device: virtual mirror for navigated spine surgery.

This paper introduces a new method for navigated spine surgery using a stereoscopic video see-through head-mounted display (HMD) and an optical tracking system. Vertebrae are segmented from volumetric CT data and visualized in-situ. A surgical drilling device is virtually extended with a mirror for intuitive planning of the drill canal, control of drill direction and insertion depth. The first designated application for the virtually extended drilling device is the preparation of canals for pedicle screw implantation in spine surgery. The objective of surgery is to install an internal fixateur for stabilization of injured vertebrae. We invited five surgeons of our partner clinic to test the system with realistic replica of lumbar vertebrae and compared the new approach with the classical, monitor-based navigation system providing three orthogonal slice views on the operation site. We measured time of procedure and scanned the drilled vertebrae with CT to verify accuracy of drilling.

Depth perception--a major issue in medical AR: evaluation study by twenty surgeons.

The idea of in-situ visualization for surgical procedures has been widely discussed in the community. While the tracking technology offers nowadays a sufficient accuracy and visualization devices have been developed that fit seamlessly into the operational workflow [1, 3], one crucial problem remains, which has been discussed already in the first paper on medical augmented reality. Even though the data is presented at the correct place, the physician often perceives the spatial position of the visualization to be closer or further because of virtual/real overlay. This paper describes and evaluates novel visualization techniques that are designed to overcome misleading depth perception of trivially superimposed virtual images on the real view. We have invited 20 surgeons to evaluate seven different visualization techniques using a head mounted display (HMD). The evaluation has been divided into two parts. In the first part, the depth perception of each kind of visualization is evaluated quantitatively. In the second part, the visualizations are evaluated qualitatively in regard to user friendliness and intuitiveness. This evaluation with a relevant number of surgeons using a state-of-the-art system is meant to guide future research and development on medical augmented reality.