Video-based 3D reconstruction, laparoscope localization and deformation recovery for abdominal minimally invasive surgery: a survey.

BACKGROUND: The intra-operative three-dimensional (3D) structure of tissue organs and laparoscope motion are the basis for many tasks in computer-assisted surgery (CAS), such as safe surgical navigation and registration of pre-operative and intra-operative data for soft tissues. METHODS: This article provides a literature review on laparoscopic video-based intra-operative techniques of 3D surface reconstruction, laparoscope localization and tissue deformation recovery for abdominal minimally invasive surgery (MIS). RESULTS: This article introduces a classification scheme based on the motions of a laparoscope and the motions of tissues. In each category, comprehensive discussion is provided on the evolution of both classic and state-of-the-art methods. CONCLUSIONS: Video-based approaches have many advantages, such as providing intra-operative information without introducing extra hardware to the current surgical platform. However, an extensive discussion on this important topic is still lacking. This survey paper is therefore beneficial for researchers in this field. Copyright © 2015 John Wiley & Sons, Ltd.

Efficient vessel feature detection for endoscopic image analysis.

Distinctive feature detection is an essential task in computer-assisted minimally invasive surgery (MIS). For special conditions in an MIS imaging environment, such as specular reflections and texture homogeneous areas, the feature points extracted by general feature point detectors are less distinctive and repeatable in MIS images. We observe that abundant blood vessels are available on tissue surfaces and can be extracted as a new set of image features. In this paper, two types of blood vessel features are proposed for endoscopic images: branching points and branching segments. Two novel methods, ridgeness-based circle test and ridgeness-based branching segment detection are presented to extract branching points and branching segments, respectively. Extensive in vivo experiments were conducted to evaluate the performance of the proposed methods and compare them with the state-of-the-art methods. The numerical results verify that, in MIS images, the blood vessel features can produce a large number of points.More importantly, those points are more robust and repeatable than the other types of feature points. In addition, due to the difference in feature types, vessel features can be combined with other general features, which makes them new tools for MIS image analysis. These proposed methods are efficient and the code and datasets are made available to the public.

Dense surface reconstruction with shadows in MIS.

Three-dimensional reconstruction of internal organ surfaces provides useful information for better control and guidance of the operations of surgical tools for minimally invasive surgery (MIS). The current reconstruction techniques using stereo cameras are still challenging due to the difficulties in correspondence matching in MIS, since there is very limited texture but significant specular reflection on organ surfaces. This paper proposes a new approach to overcome the problem by introducing weakly structured light actively casting surgical tool shadows on organ surfaces. The contribution of this paper is twofold: first, we propose a robust approach to extract shadow edges from a sequence of shadowed images; second, we develop a novel field surface interpolation (FSI) approach to obtain an accurate and dense disparity map. Our approach does not rely on texture information and is able to reconstruct accurate 3-D information by exploiting shadows from surgical tools. One advantage is that the point correspondences are directly calculated and no explicit stereo matching is required, which ensures the efficiency of the method. Another advantage is the minimum hardware requirement because only stereo cameras and a separated single-point light source are required. We evaluated the proposed approach using both phantom models and ex vivo images. Based on the experimental results, we achieved the precision of the recovered 3-D surfaces within 0.7 mm for phantom models and 1.2 mm for ex vivo images. The comparison of disparity maps indicates that with the addition of shadows, the proposed method significantly outperforms the state-of-the-art stereo algorithms for MIS.

Virtually transparent epidermal imagery (VTEI): on new approaches to in vivo wireless high-definition video and image processing.

This work first overviews a novel design, and prototype implementation, of a virtually transparent epidermal imagery (VTEI) system for laparo-endoscopic single-site (LESS) surgery. The system uses a network of multiple, micro-cameras and multiview mosaicking to obtain a panoramic view of the surgery area. The prototype VTEI system also projects the generated panoramic view on the abdomen area to create a transparent display effect that mimics equivalent, but higher risk, open-cavity surgeries. The specific research focus of this paper is on two important aspects of a VTEI system: 1) in vivo wireless high-definition (HD) video transmission and 2) multi-image processing-both of which play key roles in next-generation systems. For transmission and reception, this paper proposes a theoretical wireless communication scheme for high-definition video in situations that require extremely small-footprint image sensors and in zero-latency applications. In such situations the typical optimized metrics in communication schemes, such as power and data rate, are far less important than latency and hardware footprint that absolutely preclude their use if not satisfied. This work proposes the use of a novel Frequency-Modulated Voltage-Division Multiplexing (FM-VDM) scheme where sensor data is kept analog and transmitted via "voltage-multiplexed" signals that are also frequency-modulated. Once images are received, a novel Homographic Image Mosaicking and Morphing (HIMM) algorithm is proposed to stitch images from respective cameras, that also compensates for irregular surfaces in real-time, into a single cohesive view of the surgical area. In VTEI, this view is then visible to the surgeon directly on the patient to give an "open cavity" feel to laparoscopic procedures.

Virtually transparent epidermal imagery for laparo-endoscopic single-site surgery.

This paper presents a novel design, and prototype implementation, of a virtually transparent epidermal imagery (VTEI) system for laparo-endoscopic single-site (LESS) surgery. The system uses a network of multiple, micro wireless cameras and multiview mosaicing technique to obtain a panoramic view of the surgery area. This view provides visual feedback to surgeons with large viewing angles and areas of interest so that the surgeons can improve the safety of surgical procedures by being better aware of where the surgical instruments are relative to tissue and organs. The prototype VTEI system also projects the generated panoramic view on the abdomen area to create a transparent display effect that mimics equivalent, but higher risk, open-cavity surgeries.