On-patient see-through augmented reality based on visual SLAM.

PURPOSE: An augmented reality system to visualize a 3D preoperative anatomical model on intra-operative patient is proposed. The hardware requirement is commercial tablet-PC equipped with a camera. Thus, no external tracking device nor artificial landmarks on the patient are required. METHODS: We resort to visual SLAM to provide markerless real-time tablet-PC camera location with respect to the patient. The preoperative model is registered with respect to the patient through 4-6 anchor points. The anchors correspond to anatomical references selected on the tablet-PC screen at the beginning of the procedure. RESULTS: Accurate and real-time preoperative model alignment (approximately 5-mm mean FRE and TRE) was achieved, even when anchors were not visible in the current field of view. The system has been experimentally validated on human volunteers, in vivo pigs and a phantom. CONCLUSIONS: The proposed system can be smoothly integrated into the surgical workflow because it: (1) operates in real time, (2) requires minimal additional hardware only a tablet-PC with camera, (3) is robust to occlusion, (4) requires minimal interaction from the medical staff.

Developing a new methodology to characterize in vivo the passive mechanical behavior of abdominal wall on an animal model.

The most common surgical repair of abdominal wall hernia goes through implanting a mesh that substitutes the abdominal muscle/fascia while it is healing. To reduce the risk of relapse or possible complications, this mesh needs to mimic the mechanical behavior of the muscle/fascia, which nowadays is not fully determined. The aim of this work is to develop a methodology to characterize in vivo the passive mechanical behavior of the abdominal wall. For that, New Zealand rabbits were subjected to pneumoperitoneum tests, taking the inner pressure from 0 mmHg to 12 mmHg, values similar to those used in human laparoscopies. Animals treated were divided into two groups: healthy and herniated animals with a surgical mesh (polypropylene Surgipro(TM) Covidien) previously implanted. All experiments were recorded by a stereo rig composed of two synchronized cameras. During the postprocessing of the images, several points over the abdominal surface were tracked and their coordinates extracted for different levels of internal pressure. Starting from that, a three dimensional model of the abdominal wall was reconstructed. Pressure-displacement curves, radii of curvature and strain fields were also analysed. During the experiments, animals tissue mostly deformed during the first levels of pressure, showing the noticeable hyperelastic passive behavior of abdominal muscles. Comparison between healthy and herniated specimen displayed a strong stiffening for herniated animals in the zone where the high density mesh was situated. Cameras were able to discern this change, so this method can be used to measure the possible effect of other meshes.

Computer vision distance measurement from endoscopic sequences: prospective evaluation in laparoscopic ventral hernia repair.

BACKGROUND: Research in computer vision and mobile robotics has developed a family of popular algorithms known as Visual Simultaneous Localization And Mapping (Visual SLAM). These algorithms can provide 3D models of body cavities using the images obtained from standard monocular endoscopes. The 3D models can be used to estimate hernia defect measurements during laparoscopic ventral hernia repair (LVHR). METHODS: We conducted a descriptive and comparative prospective study to analyze results from 15 patients who underwent LVHR. Three methods of measurement were used in each patient: two classical methods (needle and tape) and a new visual SLAM measurement (VSM) method. The major and minor axes of the ellipse-shaped hernia defect were measured. RESULTS: Both axes could be measured using the VSM method in all patients except one (93%). The tape method measured 63% of the axes, but was difficult to perform because of patient comorbidities and because of limited range of motion of the laparoscopic tools. The needle method obtained 73% of measurements, because of patient comorbidities. The tape method was the most accurate (accuracy up to 0.5 cm because of tape resolution). The needle method was relatively inaccurate, with a mean error of >3 cm. The VSM method was as accurate as the tape method. The mean time taken to perform measurements was 40 s for the VSM method (range 29-60 s), 169 s for the needle method (range 66-300 s), and 186 s for the tape method (range 110-322 s). CONCLUSIONS: The needle method is relatively inaccurate and invasive. The tape method is accurate, but is not easy to perform and is relatively time consuming. The VSM method is noninvasive and fast and is as accurate as the tape method.

Visual SLAM for Handheld Monocular Endoscope.

Simultaneous localization and mapping (SLAM) methods provide real-time estimation of 3-D models from the sole input of a handheld camera, routinely in mobile robotics scenarios. Medical endoscopic sequences mimic a robotic scenario in which a handheld camera (monocular endoscope) moves along an unknown trajectory while observing an unknown cavity. However, the feasibility and accuracy of SLAM methods have not been extensively validated with human in vivo image sequences. In this work, we propose a monocular visual SLAM algorithm tailored to deal with medical image sequences in order to provide an up-to-scale 3-D map of the observed cavity and the endoscope trajectory at frame rate. The algorithm is validated over synthetic data and human in vivo sequences corresponding to 15 laparoscopic hernioplasties where accurate ground-truth distances are available. It can be concluded that the proposed procedure is: 1) noninvasive, because only a standard monocular endoscope and a surgical tool are used; 2) convenient, because only a hand-controlled exploratory motion is needed; 3) fast, because the algorithm provides the 3-D map and the trajectory in real time; 4) accurate, because it has been validated with respect to ground-truth; and 5) robust to inter-patient variability, because it has performed successfully over the validation sequences.