Augmented reality in liver surgery.

INTRODUCTION: During an operation, augmented reality (AR) enables surgeons to enrich their vision of the operating field by means of digital imagery, particularly as regards tumors and anatomical structures. While in some specialties, this type of technology is routinely ustilized, in liver surgery due to the complexity of modeling organ deformities in real time, its applications remain limited. At present, numerous teams are attempting to find a solution applicable to current practice, the objective being to overcome difficulties of intraoperative navigation in an opaque organ. OBJECTIVE: To identify, itemize and analyze series reporting AR techniques tested in liver surgery, the objectives being to establish a state of the art and to provide indications of perspectives for the future. METHODS: In compliance with the PRISMA guidelines and availing ourselves of the PubMed, Embase and Cochrane databases, we identified English-language articles published between January 2020 and January 2022 corresponding to the following keywords: augmented reality, hepatic surgery, liver and hepatectomy. RESULTS: Initially, 102 titles, studies and summaries were preselected. Twenty-eight corresponding to the inclusion criteria were included, reporting on 183patients operated with the help of AR by laparotomy (n=31) or laparoscopy (n=152). Several techniques of acquisition and visualization were reported. Anatomical precision was the main assessment criterion in 19 articles, with values ranging from 3mm to 14mm, followed by time of acquisition and clinical feasibility. CONCLUSION: While several AR technologies are presently being developed, due to insufficient anatomical precision their clinical applications have remained limited. That much said, numerous teams are currently working toward their optimization, and it is highly likely that in the short term, the application of AR in liver surgery will have become more frequent and effective. As for its clinical impact, notably in oncology, it remains to be assessed.

Unscented Kalman Filtering for Real Time Thermometry During Laser Ablation Interventions.

We present a data-assimilation Bayesian framework in the context of laser ablation for the treatment of cancer. For solving the nonlinear estimation of the tissue temperature evolving during the therapy, the Unscented Kalman Filter (UKF) predicts the next thermal status and controls the ablation process, based on sparse temperature information. The purpose of this paper is to study the outcome of the prediction model based on UKF and to assess the influence of different model settings on the framework performances. In particular, we analyze the effects of the time resolution of the filter and the number and the location of the observations. Clinical Relevance - The application of a data-assimilation approach based on limited temperature information allows to monitor and predict in real-time the thermal effects induced by thermal therapy for tumors.

Model-Based Thermometry for Laser Ablation Procedure Using Kalman Filters and Sparse Temperature Measurements.

OBJECTIVE: We implement a data assimilation Bayesian framework for the reconstruction of the spatiotemporal profile of the tissue temperature during laser irradiation. The predictions of a physical model simulating the heat transfer in the tissue are associated with sparse temperature measurements, using an Unscented Kalman Filter. METHODS: We compare a standard state-estimation filtering procedure with a joint-estimation (state and parameters) approach: whereas in the state-estimation only the temperature is evaluated, in the joint-estimation the filter corrects also uncertain model parameters (i.e., the medium thermal diffusivity, and laser beam properties). We have tested the method on synthetic temperature data, and on the temperature measured on agar-gel phantom and porcine liver with fiber optic sensors. RESULTS: The joint-estimation allows retrieving an accurate estimate of the temperature distribution with a maximal error 1.5 °C in both synthetic and liver 1D data, and 2 °C in phantom 2D data. Our approach allows also suggesting a strategy for optimizing the temperature estimation based on the positions of the sensors. Under the constraint of using only two sensors, optimal temperature estimation is obtained when one sensor is placed in proximity of the source, and the other one is non-symmetrical. CONCLUSION: The joint-estimation significantly improves the predictive capability of the physical model. SIGNIFICANCE: This work opens new perspectives on the benefit of data assimilation frameworks for laser therapy monitoring.

Blood vessel modeling for interactive simulation of interventional neuroradiology procedures.

Endovascular interventions can benefit from interactive simulation in their training phase but also during pre-operative and intra-operative phases if simulation scenarios are based on patient data. A key feature in this context is the ability to extract, from patient images, models of blood vessels that impede neither the realism nor the performance of simulation. This paper addresses both the segmentation and reconstruction of the vasculature from 3D Rotational Angiography data, and adapted to simulation: An original tracking algorithm is proposed to segment the vessel tree while filtering points extracted at the vessel surface in the vicinity of each point on the centerline; then an automatic procedure is described to reconstruct each local unstructured point set as a skeleton-based implicit surface (blobby model). The output of successively applying both algorithms is a new model of vasculature as a tree of local implicit models. The segmentation algorithm is compared with Multiple Hypothesis Testing (MHT) algorithm (Friman et al., 2010) on patient data, showing its greater ability to track blood vessels. The reconstruction algorithm is evaluated on both synthetic and patient data and demonstrate its ability to fit points with a subvoxel precision. Various tests are also reported where our model is used to simulate catheter navigation in interventional neuroradiology. An excellent realism, and much lower computational costs are reported when compared to triangular mesh surface models.

Ex vivo model of congenital cytomegalovirus infection and new combination therapies.

INTRODUCTION: Congenital human cytomegalovirus (HCMV) infection is a major public health problem due to severe sequelae in the fetus and newborns. Currently, due to their toxicity anti-CMV treatments cannot be administered to pregnant women. We thus developed an ex vivo model of 1(st) trimester placental CMV infection to observe the route of infection across the placenta and to test the efficacy of various new drugs targeting different stages of viral cycle. METHODS: After validation of the viability of floating villi explants by ELISA ß-HCG, the kinetics of placental infection were determined by immunochemistry and qPCR in this ex vivo model. Antiviral susceptibility was determined in vitro using focus reduction assay and by qPCR in the ex vivo model. RESULTS: The ex vivo model showed viral infection in trophoblasts and mesenchymal space of floating villi. In vitro, antiviral combinations of maribavir with baïcalein or artesunate inhibited viral infection by more than 90%. On the other hand, in ex vivo model, infection was reduced by 40% in presence of maribavir and artesunate. The synergistic effect observed in vitro was not observed ex vivo. DISCUSSION: This model allowed us to understand the CMV spread in 1(st) trimester floating villi better and to analyze the anti-CMV efficacy and toxicity of new drugs that could be administered to pregnant women, either alone or in combination. CONCLUSIONS: Such an ex vivo model could be applied to other viruses such as rubella or parvovirus B19 and in new drug development.

Simulation of pneumoperitoneum for laparoscopic surgery planning.

Laparoscopic surgery planning is usually realized on a preoperative image that does not correspond to the operating room conditions. Indeed, the patient undergoes gas insufflation (pneumoperitoneum) to allow instrument manipulation inside the abdomen. This insufflation moves the skin and the viscera so that their positions do no longer correspond to the preoperative image, reducing the benefit of surgical planning, more particularly for the trocar positioning step. A simulation of the pneumoperitoneum influence would thus improve the realism and the quality of the surgical planning. We present in this paper a method to simulate the movement of skin and viscera due to the pneumoperitoneum. Our method requires a segmented preoperative 3D medical image associated to realistic biomechanical parameters only. The simulation is performed using the SOFA simulation engine. The results were evaluated using computed tomography [CT] images of two pigs, before and after pneumoperitoneum. Results show that our method provides a very realistic estimation of skin, viscera and artery positions with an average error within 1 cm.

Local implicit modeling of blood vessels for interactive simulation.

In the context of computer-based simulation, contact management requires an accurate, smooth, but still efficient surface model for the blood vessels. A new implicit model is proposed, consisting of a tree of local implicit surfaces generated by skeletons (blobby models). The surface is reconstructed from data points by minimizing an energy, alternating with an original blob selection and subdivision scheme. The reconstructed models are very efficient for simulation and were shown to provide a sub-voxel approximation of the vessel surface on 5 patients.

Constraint-Based Haptic Rendering of Multirate Compliant Mechanisms.

The paper is dedicated to haptic rendering of complex physics-based environment in the context of surgical simulation. A new unified formalism for modeling the mechanical interactions between medical devices and anatomical structures and for computing accurately the haptic force feedback is presented. The approach deals with the mechanical interactions using appropriate force and/or motion transmission models named compliant mechanisms. These mechanisms are formulated as a constraint-based problem that is solved in two separate threads running at different frequencies. The first thread processes the whole simulation including the soft-tissue deformations, whereas the second one only deals with computer haptics. This method builds a bridge between the so-called virtual mechanisms (that were proposed for haptic rendering of rigid bodies) and intermediate representations (used for rendering of complex simulations). With this approach, it is possible to describe the specific behavior of various medical devices while relying on a unified method for solving the mechanical interactions between deformable objects and haptic rendering. The technique is demonstrated in interactive simulation of flexible needle insertion through soft anatomical structures with force feedback.

Direct genotyping of cytomegalovirus envelope glycoproteins from toddler's saliva samples.

BACKGROUND: The polymorphism of genes encoding CMV envelope protein is used for strain classification and may influence pathogenesis and/or infectivity. CMV genotyping is usually based on sequencing or acrylamide gel-RFLP, but these methods are not suited to rapid screening of large populations. OBJECTIVES: We developed a high-throughput method to analyze CMV strains diversity and to detect multiple-strain infection in a large population of toddlers (six daycare centers (DCC) and an emergency unit (EU)). METHODS: We developed a new PCR-RFLP method coupled with capillary electrophoresis fragment detection for UL55-gB, UL75-gH and UL73-gN genotyping. To detect gB recombinants, gpUL55 typing was applied to two variable zones (NTerminal and central). We applied this method to 212 CMV-positive saliva samples and controlled the results by direct sequencing of PCR products. RESULTS: We identified 112 strains, that fell into eight groups in UL55-gB, two groups in UL75-gH, and seven groups in UL73-gN. The 79 samples from the emergency unit contained 30 strains, 28 children harboring 2 strains. The samples (n=133) from the six daycare centers contained respectively 4, 1, 6, 1 and 11 strains. Fifteen percent of strains were UL55-gB recombinants. CONCLUSION: Our new method can simultaneously determine gB, gH and gN genotypes and offers more precise classification of CMV strains than previous RFLP-based methods. This could constitute the basis for a new classification, particularly in UL55-gB. Easy direct identification of multiple strains and recombinants in pathological samples could facilitate large epidemiologic studies.

SOFA--an open source framework for medical simulation.

SOFA is a new open source framework primarily targeted at medical simulation research. Based on an advanced software architecture, it allows to (1) create complex and evolving simulations by combining new algorithms with algorithms already included in SOFA; (2) modify most parameters of the simulation--deformable behavior, surface representation, solver, constraints, collision algorithm, etc.--by simply editing an XML file; (3) build complex models from simpler ones using a scene-graph description; (4) efficiently simulate the dynamics of interacting objects using abstract equation solvers; and (5) reuse and easily compare a variety of available methods. In this paper we highlight the key concepts of the SOFA architecture and illustrate its potential through a series of examples.

New approaches to catheter navigation for interventional radiology simulation.

For over 20 years, interventional methods have improved the outcomes of patients with cardiovascular disease. However, these procedures require an intricate combination of visual and tactile feedback and extensive training. In this paper, we describe a series of novel approaches that have led to the development of a high-fidelity simulation system for interventional neuroradiology. In particular, we focus on a new approach for real-time deformation of devices such as catheters and guidewires during navigation inside complex vascular networks. This approach combines a real-time incremental Finite Element Model (FEM), an optimization strategy based on substructure decomposition, and a new method for handling collision response in situations where the number of contact points is very large. We also briefly describe other aspects of the simulation system, from patient-specific segmentation to the simulation of contrast agent propagation and fast volume-rendering techniques for generating synthetic X-ray images in real time. Although currently targeted at stroke therapy, our results are applicable to the simulation of any interventional radiology procedure.

New approaches to catheter navigation for interventional radiology simulation.

For over 20 years, interventional methods have improved the outcomes of patients with cardiovascular disease. However, these procedures require an intricate combination of visual and tactile feedback and extensive training periods. In this paper, we describe a series of novel approaches that have lead to the development of a high-fidelity simulation system for interventional neuroradiology. In particular we focus on a new approach for real-time deformation of devices such as catheters and guidewires during navigation inside complex vascular networks. This approach combines a real-time incremental Finite Element Model, an optimization strategy based on substructure decomposition, and a new method for handling collision response in situations where the number of contacts points is very large. We also briefly describe other aspects of the simulation system, from patient-specific segmentation to the simulation of contrast agent propagation and fast volume rendering techniques for generating synthetic X-ray images in real-time.

Computer-enhanced laparoscopic training system (CELTS): bridging the gap.

BACKGROUND: There is a large and growing gap between the need for better surgical training methodologies and the systems currently available for such training. In an effort to bridge this gap and overcome the disadvantages of the training simulators now in use, we developed the Computer-Enhanced Laparoscopic Training System (CELTS). METHODS: CELTS is a computer-based system capable of tracking the motion of laparoscopic instruments and providing feedback about performance in real time. CELTS consists of a mechanical interface, a customizable set of tasks, and an Internet-based software interface. The special cognitive and psychomotor skills a laparoscopic surgeon should master were explicitly defined and transformed into quantitative metrics based on kinematics analysis theory. A single global standardized and task-independent scoring system utilizing a z-score statistic was developed. Validation exercises were performed. RESULTS: The scoring system clearly revealed a gap between experts and trainees, irrespective of the task performed; none of the trainees obtained a score above the threshold that distinguishes the two groups. Moreover, CELTS provided educational feedback by identifying the key factors that contributed to the overall score. Among the defined metrics, depth perception, smoothness of motion, instrument orientation, and the outcome of the task are major indicators of performance and key parameters that distinguish experts from trainees. Time and path length alone, which are the most commonly used metrics in currently available systems, are not considered good indicators of performance. CONCLUSION: CELTS is a novel and standardized skills trainer that combines the advantages of computer simulation with the features of the traditional and popular training boxes. CELTS can easily be used with a wide array of tasks and ensures comparability across different training conditions. This report further shows that a set of appropriate and clinically relevant performance metrics can be defined and a standardized scoring system can be designed.

Designing a computer-based simulator for interventional cardiology training.

Interventional cardiology training traditionally involves one-on-one experience following a master-apprentice model, much as other procedural disciplines. Development of a realistic computer-based training system that includes hand-eye coordination, catheter and guide wire choices, three-dimensional anatomic representations, and an integrated learning system is desirable, in order to permit learning to occur safely, without putting patients at risk. Here we present the first report of a PC-based simulator that incorporates synthetic fluoroscopy, real-time three-dimensional interactive anatomic display, and selective right- and left-sided coronary catheterization and angiography using actual catheters. Significant learning components also are integrated into the simulator.

ICTS, an interventional cardiology training system.

In this article, we present an Interventional Cardiology Training System developed by the Medical Application Group at Mitsubishi Electric in collaboration with the Center for Innovative Minimally Invasive Therapy. The core of the ICTS is a computer simulation of interventional cardiology catheterization. This simulation integrates clinical expertise, research in learning, and technical innovations to create a realistic simulated environment. The goal of this training system is to augment the training of new cardiology fellows as well as to introduce cardiologists to new devices and procedures. To achieve this goal, both the technical components and the educational content of the ICTS bring new and unique features: a simulated fluoroscope, a physics model of a catheter, a haptic interface, a fluid flow simulation combined with a hemodynamic model and a learning system integrated in a user interface. The simulator is currently able to generate--in real-time--high quality x-ray images from a 3D anatomical model of the thorax, including a beating heart and animated lungs. The heart and lung motion is controlled by the hemodynamic model, which also computes blood pressure and EKG. The blood flow is then calculated according to the blood pressure and blood vessel characteristics. Any vascular tool, such as a catheter, guide wire or angioplasty balloon can be represented and accurately deformed by the flexible tool physics model. The haptics device controls the tool and provides appropriate feedback when contact with a vessel wall is detected. When the catheter is in place, a contrast agent can be injected into the coronary arteries; blood and contrast mixing is computed and a visual representation of the angiogram is displayed by the x-ray renderer. By bringing key advances in the area of medical simulation--with the real-time x-ray renderer for instance--and by integrating in a single system both high quality simulation and learning tools, the ICTS opens new perspectives for computer based training systems.

Efficient linear elastic models of soft tissues for real-time surgery simulation.

In this paper, we describe the basic components of a surgery simulator prototype developed at INRIA. We present two physical models which are well suited for surgery simulation. These models are based on linear elasticity theory and finite elements modeling. The former model can deforme large tetrahedral meshes in real-time but does not allow any topological changes. On the contrary, the latter biomechanical model can simulate the cutting and tearing of soft tissue but must have a limited number of vertices to run in real-time. We propose a method for combining these two approaches into a hybrid model which may allow real time deformations and cuttings of large enough anatomical structures.

Virtual reality applied to hepatic surgery simulation: the next revolution.

OBJECTIVE: This article describes a preliminary work on virtual reality applied to liver surgery and discusses the repercussions of assisted surgical strategy and surgical simulation on tomorrow's surgery. SUMMARY BACKGROUND DATA: Liver surgery is considered difficult because of the complexity and variability of the organ. Common generic tools for presurgical medical image visualization do not fulfill the requirements for the liver, restricting comprehension of a patient's specific liver anatomy. METHODS: Using data from the National Library of Medicine, a realistic three-dimensional image was created, including the envelope and the four internal arborescences. A computer interface was developed to manipulate the organ and to define surgical resection planes according to internal anatomy. The first step of surgical simulation was implemented, providing the organ with real-time deformation computation. RESULTS: The three-dimensional anatomy of the liver could be clearly visualized. The virtual organ could be manipulated and a resection defined depending on the anatomic relations between the arborescences, the tumor, and the external envelope. The resulting parts could also be visualized and manipulated. The simulation allowed the deformation of a liver model in real time by means of a realistic laparoscopic tool. CONCLUSIONS: Three-dimensional visualization of the organ in relation to the pathology is of great help to appreciate the complex anatomy of the liver. Using virtual reality concepts (navigation, interaction, and immersion), surgical planning, training, and teaching for this complex surgical procedure may be possible. The ability to practice a given gesture repeatedly will revolutionize surgical training, and the combination of surgical planning and simulation will improve the efficiency of intervention, leading to optimal care delivery.

[A new concept in surgery of the digestive tract: surgical procedure assisted by computer, from virtual reality to telemanipulation]. / Un nouveau concept en chirurgie digestive: la procédure chirurgicale assistée par ordinateur, de la réalité virtuelle à la télémanipulation.

Surgical simulation increasingly appears to be an essential aspect of tomorrow's surgery. The development of a hepatic surgery simulator is an advanced concept calling for a new writing system which will transform the medical world: virtual reality. Virtual reality extends the perception of our five senses by representing more than the real state of things by the means of computer sciences and robotics. It consists of three concepts: immersion, navigation and interaction. Three reasons have led us to develop this simulator: the first is to provide the surgeon with a comprehensive visualisation of the organ. The second reasons is to allow for planning and surgical simulation that could be compared with the detailed flight-plan for a commercial jet pilot. The third lies in the fact that virtual reality is an integrated part of the concept of computer assisted surgical procedure. The project consists of a sophisticated simulator which must include five requirements: a) visual fidelity, b) interactivity, c) physical properties, d) physiological properties, e) sensory input and output. In this report we describe how to obtain a realistic 3D model of the liver from bi-dimensional 2D medical images for anatomical and surgical training. The introduction of a tumor and the consequent planning and virtual resection is also described, as are force feedback and real-time interaction.

A fully three-dimensional method for facial reconstruction based on deformable models.

Two facial models corresponding to two deceased subjects have been manually created and the two corresponding skulls have been dissected and skeletonized. These pairs of skull/ facial data have been scanned with a CT scanner, and the computed geometric three-dimensional models of both skulls and facial tissue have been built. One set of skull/facial data will be used as a reference set whereas the second set is used as ground truth for validating our method. After a semi-automatic face-skull registration, we apply an original computing global parametric transformation T that turns the reference skull into the skull to be reconstructed. This algorithm is based upon salient lines of the skull called crest lines: more precisely the crest lines of the first skull are matched to the crest lines of the second skull by an iterative closest point algorithm. Then we apply this algorithm to the reference face to obtain the "unknown" face to be reconstructed. The reliability and difficulties of this original technique are then discussed.

[A new concept in digestive surgery: the computer assisted surgical procedure, from virtual reality to telemanipulation]. / Un nouveau concept en chirurgie digestive: la procédure chirurgicale assistée par ordinateur, de la réalité virtuelle à la télémanipulation.

Surgical simulation increasingly appears to be an essential aspect of tomorrow's surgery. The development of a hepatic surgery simulator is an advanced concept calling for a new writing system which will transform the medical world: virtual reality. Virtual reality extends the perception of our five senses by representing more than the real state of things by the means of computer sciences and robotics. It consists of three concepts: immersion, navigation and interaction. Three reasons have led us to develop this simulator: the first is to provide the surgeon with a comprehensive visualisation of the organ. The second reason is to allow for planning and surgical simulation that could be compared with the detailed flight-plan for a commercial jet pilot. The third lies in the fact that virtual reality is an integrated part of the concept of computer assisted surgical procedure. The project consists of a sophisticated simulator which has to include five requirements: visual fidelity, interactivity, physical properties, physiological properties, sensory input and output. In this report we will describe how to get a realistic 3D model of the liver from bi-dimensional 2D medical images for anatomical and surgical training. The introduction of a tumor and the consequent planning and virtual resection is also described, as are force feedback and real-time interaction.