En-face optical coherence tomography/fluorescence endomicroscopy for minimally invasive imaging using a robotic scanner.

We report a compact rigid instrument capable of delivering en-face optical coherence tomography (OCT) images alongside (epi)-fluorescence endomicroscopy (FEM) images by means of a robotic scanning device. Two working imaging channels are included: one for a one-dimensional scanning, forward-viewing OCT probe and another for a fiber bundle used for the FEM system. The robotic scanning system provides the second axis of scanning for the OCT channel while allowing the field of view (FoV) of the FEM channel to be increased by mosaicking. The OCT channel has resolutions of 25 / 60 µm (axial/lateral) and can provide en-face images with an FoV of 1.6 × 2.7 mm2. The FEM channel has a lateral resolution of better than 8 µm and can generate an FoV of 0.53 × 3.25 mm2 through mosaicking. The reproducibility of the scanning was determined using phantoms to be better than the lateral resolution of the OCT channel. Combined OCT and FEM imaging were validated with ex-vivo ovine and porcine tissues, with the instrument mounted on an arm to ensure constant contact of the probe with the tissue. The OCT imaging system alone was validated for in-vivo human dermal imaging with the handheld instrument. In both cases, the instrument was capable of resolving fine features such as the sweat glands in human dermal tissue and the alveoli in porcine lung tissue.

Intraoperative Robotic-Assisted Large-Area High-Speed Microscopic Imaging and Intervention.

OBJECTIVE: Probe-based confocal endomicroscopy is an emerging high-magnification optical imaging technique that provides in vivo and in situ cellular-level imaging for real-time assessment of tissue pathology. Endomicroscopy could potentially be used for intraoperative surgical guidance, but it is challenging to assess a surgical site using individual microscopic images due to the limited field-of-view and difficulties associated with manually manipulating the probe. METHODS: In this paper, a novel robotic device for large-area endomicroscopy imaging is proposed, demonstrating a rapid, but highly accurate, scanning mechanism with image-based motion control, which is able to generate histology-like endomicroscopy mosaics. The device also includes, for the first time in robotic-assisted endomicroscopy, the capability to ablate tissue without the need for an additional tool. RESULTS: The device achieves preprogrammed trajectories with positioning accuracy of less than 30 [Formula: see text], while the image-based approach demonstrated that it can suppress random motion disturbances up to [Formula: see text]. Mosaics are presented from a range of ex vivo human and animal tissues, over areas of more than [Formula: see text], scanned in approximate [Formula: see text]. CONCLUSION: This paper demonstrates the potential of the proposed instrument to generate large-area, high-resolution microscopic images for intraoperative tissue identification and margin assessment. SIGNIFICANCE: This approach presents an important alternative to current histology techniques, significantly reducing the tissue assessment time, while simultaneously providing the capability to mark and ablate suspicious areas intraoperatively.

Can surgical simulation be used to train detection and classification of neural networks?

Computer-assisted interventions (CAI) aim to increase the effectiveness, precision and repeatability of procedures to improve surgical outcomes. The presence and motion of surgical tools is a key information input for CAI surgical phase recognition algorithms. Vision-based tool detection and recognition approaches are an attractive solution and can be designed to take advantage of the powerful deep learning paradigm that is rapidly advancing image recognition and classification. The challenge for such algorithms is the availability and quality of labelled data used for training. In this Letter, surgical simulation is used to train tool detection and segmentation based on deep convolutional neural networks and generative adversarial networks. The authors experiment with two network architectures for image segmentation in tool classes commonly encountered during cataract surgery. A commercially-available simulator is used to create a simulated cataract dataset for training models prior to performing transfer learning on real surgical data. To the best of authors' knowledge, this is the first attempt to train deep learning models for surgical instrument detection on simulated data while demonstrating promising results to generalise on real data. Results indicate that simulated data does have some potential for training advanced classification methods for CAI systems.

Force adaptive robotically assisted endomicroscopy for intraoperative tumour identification.

PURPOSE: For effective tumour margin definition for cancer surgery, there is an increasing demand for the development of real-time intraoperative tissue biopsy techniques. Recent advances in miniaturized biophotonics probes have permitted the development of endomicroscopy techniques that are clinically attractive. With these approaches, cellular-level imaging can be achieved through millimetre-scale flexible probes and be performed in real-time, in vivo and in situ. Due to the limited field of view and flexibility of these probes, however, large area tissue coverage for acquiring histology-like images over complex three-dimensional surfaces is challenging. This is particularly the case because current surgical robots, such as the Da Vinci ®, lack haptic feedback, making it difficult to maintain optimum tissue contact when these probes are deployed in vivo. METHODS: This paper proposes a simple force-controlled pick-up probe that can be integrated with the Da Vinci instruments for intraoperative endomicroscopy imaging. The device uses a new low-friction air bearing with adaptive axial force control to maintain constant contact between the tissue and the imaging probe, facilitating microscopy scans over complex surfaces. Detailed ex vivo user experiments have been conducted to demonstrate the effectiveness of the technique. RESULTS: The adaptive probe mount could achieve consistent low-magnitude probe-sample contact forces compared with a rigid mount. In the user study, the adaptive probe combined with a high frame rate endomicroscopy system allowed larger mosaics to be generated over curved surfaces. CONCLUSIONS: The device can improve the performance of large area mosaicking over complex 3D surfaces with improved handling and intraoperative control.

Color reflectance fiber bundle endomicroscopy without back-reflections.

Coherent fiber imaging bundles can be used as passive probes for reflectance-mode endomicroscopy providing that the back-reflections from the fiber ends are efficiently rejected. We describe an approach specific to widefield endomicroscopy in which light is injected into a leached fiber bundle near the distal end, thereby avoiding reflections from the proximal face. We use this method to demonstrate the color widefield reflectance endomicroscopy of ex vivo animal tissue.

Implicit Active Constraints for Robot-Assisted Arthroscopy.

This paper presents an Implicit Active Constraints control framework for robot-assisted minimally invasive surgery. It extends on current frameworks by prescribing the external constraints implicitly from the operator motion, forgoing the need for pre-operative imaging; the constraints are defined in situ so as to avoid the use of invasive fiducial markers. A hands-on cooperatively-controlled robotic platform, comprising of a surgical instrument and a compliant manipulator, has been designed for an arthroscopic procedure. The surgical platform is capable of constraining the pose of the instrument so as to ensure it passes through the incision point and does not cause trauma to the surrounding tissue. A flexible arthroscopic instrument is designed and its use is investigated to enlarge reachable and dexterous workspace, increasing the accessibility to the target anatomy. The behaviour of the flexible instrument is analysed. A detailed performance analysis is conducted on a group of subjects for validating the control framework, simulating a minimally invasive arthroscopic procedure. Results demonstrate a statistically significant enhancement in the control ergonomics as well as the accuracy and safety of the procedure.