[Team training and assessment in mixed reality-based simulated operating room : Current state of research in the field of simulation in spine surgery exemplified by the ATMEOS project]. / Teamtraining und Assessment im Mixed-reality-basierten simulierten OP : Aktueller Stand der Forschung im Bereich Simulation in der Wirbelsäulenchirurgie am Beispiel des Projekts ATMEOS.

Surgical simulators are being increasingly used as an attractive alternative to clinical training in addition to conventional animal models and human specimens. Typically, surgical simulation technology is designed for the purpose of teaching technical surgical skills (so-called task trainers). Simulator training in surgery is therefore in general limited to the individual training of the surgeon and disregards the participation of the rest of the surgical team. The objective of the project Assessment and Training of Medical Experts based on Objective Standards (ATMEOS) is to develop an immersive simulated operating room environment that enables the training and assessment of multidisciplinary surgical teams under various conditions. Using a mixed reality approach, a synthetic patient model, real surgical instruments and radiation-free virtual X­ray imaging are combined into a simulation of spinal surgery. In previous research studies, the concept was evaluated in terms of realism, plausibility and immersiveness. In the current research, assessment measurements for technical and non-technical skills are developed and evaluated. The aim is to observe multidisciplinary surgical teams in the simulated operating room during minimally invasive spinal surgery and objectively assess the performance of the individual team members and the entire team. Moreover, the effectiveness of training methods and surgical techniques or success critical factors, e. g. management of crisis situations, can be captured and objectively assessed in the controlled environment.

[3D augmented reality visualization for navigated osteosynthesis of pelvic fractures]. / "3D-augmented-reality"-Visualisierung für die navigierte Osteosynthese von Beckenfrakturen.

BACKGROUND: Despite great advances in the development of hardware and software components, surgical navigation systems have only seen limited use in current clinical settings due to their reported complexity, difficulty of integration into clinical workflows and questionable advantages over traditional imaging modalities. OBJECTIVES: Development of augmented reality (AR) visualization for surgical navigation without the need for infrared (IR) tracking markers and comparison of the navigation system to conventional imaging. MATERIAL AND METHODS: Novel navigation system combining a cone beam computed tomography (CBCT) capable C­arm with a red-green-blue depth (RGBD) camera. Testing of the device by Kirschner wire (K-wire) placement in phantoms and evaluation of the necessary operating time, number of fluoroscopic images and overall radiation dose were compared to conventional x­ray imaging. RESULTS: We found a significant reduction of the required time, number of fluoroscopic images and overall radiation dose in 3D AR navigation in comparison to x­ray imaging. CONCLUSION: Our AR navigation using RGBD cameras offers a flexible and intuitive visualization of the operating field for the navigated osteosynthesis without IR tracking markers, enabling surgeons to complete operations quicker and with a lower radiation exposure to the patient and surgical staff.

De la imagen intervencionista a la guía intraoperatoria: nuevas perspectivas combinando herramientas avanzadas y navegación con la cirugía radioguiada / From interventionist imaging to intraoperative guidance: new perspectives by combining advanced tools and navigation with radio-guided surgery

La integración de tecnologías de imagen médica en los enfoques diagnósticos y terapéuticos puede proporcionar una perspectiva preoperatoria tanto en los aspectos anatómicos (tomografía computarizada, resonancia magnética o ecografía) como funcional (tomografía computarizada de emisión de fotón único, tomografía por emisión de positrones, linfogammagrafía o imagen óptica). Además, algunas modalidades de imagen se utilizan también en un entorno intervencionista (tomografía computarizada, ecografía, imágenes gammagráficas o imágenes ópticas), donde proporcionan al cirujano información en tiempo real durante el procedimiento. En la actualidad son factibles diversas herramientas y enfoques metodológicos para la navegación guiada por imágenes en la cirugía del cáncer. Con el desarrollo de nuevos trazadores y dispositivos portátiles de imagen, estos avances reforzarán el papel de la imagen molecular intervencionista (AU)

The integration of medical imaging technologies into diagnostic and therapeutic approaches can provide a preoperative insight into both anatomical (e.g. using computed tomography, magnetic resonance imaging, or ultrasound), as well as functional aspects (e.g. using single photon emission computed tomography, positron emission tomography, lymphoscintigraphy, or optical imaging). Moreover, some imaging modalities are also used in an interventional setting (e.g. computed tomography, ultrasound, gamma or optical imaging) where they provide the surgeon with real-time information during the procedure. Various tools and approaches for image-guided navigation in cancer surgery are becoming feasible today. With the development of new tracers and portable imaging devices, these advances will reinforce the role of interventional molecular imaging (AU)

From interventionist imaging to intraoperative guidance: New perspectives by combining advanced tools and navigation with radio-guided surgery. / De la imagen intervencionista a la guía intraoperatoria: nuevas perspectivas combinando herramientas avanzadas y navegación con la cirugía radioguiada.

The integration of medical imaging technologies into diagnostic and therapeutic approaches can provide a preoperative insight into both anatomical (e.g. using computed tomography, magnetic resonance imaging, or ultrasound), as well as functional aspects (e.g. using single photon emission computed tomography, positron emission tomography, lymphoscintigraphy, or optical imaging). Moreover, some imaging modalities are also used in an interventional setting (e.g. computed tomography, ultrasound, gamma or optical imaging) where they provide the surgeon with real-time information during the procedure. Various tools and approaches for image-guided navigation in cancer surgery are becoming feasible today. With the development of new tracers and portable imaging devices, these advances will reinforce the role of interventional molecular imaging.

The 2014 liver ultrasound tracking benchmark.

The Challenge on Liver Ultrasound Tracking (CLUST) was held in conjunction with the MICCAI 2014 conference to enable direct comparison of tracking methods for this application. This paper reports the outcome of this challenge, including setup, methods, results and experiences. The database included 54 2D and 3D sequences of the liver of healthy volunteers and tumor patients under free breathing. Participants had to provide the tracking results of 90% of the data (test set) for pre-defined point-landmarks (healthy volunteers) or for tumor segmentations (patient data). In this paper we compare the best six methods which participated in the challenge. Quantitative evaluation was performed by the organizers with respect to manual annotations. Results of all methods showed a mean tracking error ranging between 1.4 mm and 2.1 mm for 2D points, and between 2.6 mm and 4.6 mm for 3D points. Fusing all automatic results by considering the median tracking results, improved the mean error to 1.2 mm (2D) and 2.5 mm (3D). For all methods, the performance is still not comparable to human inter-rater variability, with a mean tracking error of 0.5-0.6 mm (2D) and 1.2-1.8 mm (3D). The segmentation task was fulfilled only by one participant, resulting in a Dice coefficient ranging from 76.7% to 92.3%. The CLUST database continues to be available and the online leader-board will be updated as an ongoing challenge.

[Intraoperative augmented reality visualization. Current state of development and initial experiences with the CamC]. / Intraoperative "Augmented-reality-Visualisierung". Aktueller Stand der Entwicklung und erste Erfahrungen mit dem CamC.

The intraoperative application of augmented reality (AR) has so far mainly taken place in the field of endoscopy. Here, the camera image of the endoscope was augmented by computer graphics derived mostly from preoperative imaging. Due to the complex setup and operation of the devices, they have not yet become part of routine clinical practice. The Camera Augmented Mobile C-arm (CamC) that extends a classic C-arm by a video camera and mirror construction is characterized by its uncomplicated handling. It combines its video live stream geometrically correct with the acquired X-ray. The clinical application of the device in 43 cases showed the strengths of the device in positioning for X-ray acquisition, incision placement, K-wire placement, and instrument guidance. With its new function and the easy integration into the OR workflow of any procedure that requires X-ray imaging, the CamC has the potential to become the first widely used AR technology for orthopedic and trauma surgery.

"Eye-tracking" for assessment of image perception in gastrointestinal endoscopy with narrow-band imaging compared with white-light endoscopy.

BACKGROUND AND STUDY AIMS: Narrow-band imaging (NBI) is a new imaging methodology for improving the detection rate of gastrointestinal lesions. We aimed to evaluate perception of images by NBI and corresponding standard white-light-endoscopy (WLE) using a computer-guided eye-tracking system. METHODS: A total of 23 NBI images of various lesions with the 23 corresponding WLE images were assessed in random order by 18 subjects with various endoscopy experience. Before evaluation, a teaching set of three NBI and corresponding WLE images was shown to highlight the characteristics of lesions. An eye-tracking system (Tobii X series with integrated 17-inch monitor) was used to record the eye movements of the subjects while they examined respective images. The following parameters were measured: total time spent on image, time until first fixation of lesion, total number of fixations per image and per lesion, and number of fixations until finding the lesion. RESULTS: In total, 828 experiments were conducted. Lesions could not be detected in 6.5 % (NBI) and 4.1 % (WLE) of images ( P = NS). The total number of fixations and total time spent on respective figures as a whole were significantly greater for NBI images compared with WLE images ( P < 0.003). However, the number of fixations until the lesion was found, the number of fixations on the lesion, and the time until first fixation of the lesion did not differ between the two image groups ( P > 0.1). CONCLUSION: This is the first study using eye tracking to evaluate image perception in gastrointestinal endoscopy. Significant differences in the interpretation of NBI and WLE images were observed, which may be relevant for the detection and characterization of lesions during endoscopy.

Informatics and medicine--from molecules to populations.

OBJECTIVES: To clarify challenges and research topics for informatics in health and to describe new approaches for interdisciplinary collaboration and education. METHODS: Research challenges and possible solutions were elaborated by scientists of two universities using an interdisciplinary approach, in a series of meetings over several months. RESULTS AND CONCLUSION: In order to translate scientific results from bench to bedside and further into an evidence-based and efficient health system, intensive collaboration is needed between experts from medicine, biology, informatics, engineering, public health, as well as social and economic sciences. Research challenges can be attributed to four areas: bioinformatics and systems biology, biomedical engineering and informatics, health informatics and individual healthcare, and public health informatics. In order to bridge existing gaps between different disciplines and cultures, we suggest focusing on interdisciplinary education, taking an integrative approach and starting interdisciplinary practice at early stages of education.

A boosted segmentation method for surgical workflow analysis.

As demands on hospital efficiency increase, there is a stronger need for automatic analysis, recovery, and modification of surgical workflows. Even though most of the previous work has dealt with higher level and hospital-wide workflow including issues like document management, workflow is also an important issue within the surgery room. Its study has a high potential, e.g., for building context-sensitive operating rooms, evaluating and training surgical staff, optimizing surgeries and generating automatic reports. In this paper we propose an approach to segment the surgical workflow into phases based on temporal synchronization of multidimensional state vectors. Our method is evaluated on the example of laparoscopic cholecystectomy with state vectors representing tool usage during the surgeries. The discriminative power of each instrument in regard to each phase is estimated using AdaBoost. A boosted version of the Dynamic Time Warping (DTW) algorithm is used to create a surgical reference model and to segment a newly observed surgery. Full cross-validation on ten surgeries is performed and the method is compared to standard DTW and to Hidden Markov Models.

Recovering the X-ray projection geometry for three-dimensional tomographic reconstruction with additional sensors: attached camera versus external navigation system.

Three-dimensional tomographic reconstruction using intra-operative mobile C-arms could provide physicians with new and exciting tools for image-guided surgery. Recovery of the projection geometry of mobile X-ray systems is a crucial step for such reconstruction procedures. Recent work on medical imaging describes the use of optical or electro-magnetic sensor systems in order to navigate surgical instruments. These systems can also be used for the estimation of C-arm motion, and therefore for the recovery of the projection geometry of the X-ray C-arm. In this case, the mathematical problem that needs to be solved is equivalent to the hand-eye calibration well studied by both the computer vision and robotics community. We first study the recovery of the motion and projection geometry using five different hand-eye calibration methods proposed in the literature. The optical navigation system POLARIS from Northern Digital Inc. was used in our experiments. The results of the estimated motion and projection geometry using the five hand-eye calibration methods are compared with the same results obtained using an off-the-shelf CCD camera attached to the mobile C-arm. The experimental results include three-dimensional tomographic reconstruction results using our mobile C-arm. We show that even though the motion of the C-arm is more precisely recovered using the navigation system, the projection geometry is better estimated using the attached CCD camera.

Enhanced 3-D-reconstruction algorithm for C-arm systems suitable for interventional procedures.

Increasingly, three-dimensional (3-D) imaging technologies are used in medical diagnosis, for therapy planning, and during interventional procedures. We describe the possibilities of fast 3-D-reconstruction of high-contrast objects with high spatial resolution from only a small series of two-dimensional (2-D) planar radiographs. The special problems arising from the intended use of an open, mechanically unstable C-arm system are discussed. For the description of the irregular sampling geometry, homogeneous coordinates are used thoroughly. The well-known Feldkamp algorithm is modified to incorporate corresponding projection matrices without any decomposition into intrinsic and extrinsic parameters. Some approximations to speed up the whole reconstruction procedure and the tradeoff between image quality and computation time are also considered. Using standard hardware the reconstruction of a 256(3) cube is now possible within a few minutes, a time that is acceptable during interventions. Examples for cranial vessel imaging from some clinical test installations will be shown as well as promising results for bone imaging with a laboratory C-arm system.