Semi-Automated Extraction of Lens Fragments via a Surgical Robot Using Semantic Segmentation of OCT Images with Deep Learning - Experimental Results in ex vivo Animal Model.

The overarching goal of this work is to demonstrate the feasibility of using optical coherence tomography (OCT) to guide a robotic system to extract lens fragments from ex vivo pig eyes. A convolutional neural network (CNN) was developed to semantically segment four intraocular structures (lens material, capsule, cornea, and iris) from OCT images. The neural network was trained on images from ten pig eyes, validated on images from eight different eyes, and tested on images from another ten eyes. This segmentation algorithm was incorporated into the Intraocular Robotic Interventional Surgical System (IRISS) to realize semi-automated detection and extraction of lens material. To demonstrate the system, the semi-automated detection and extraction task was performed on seven separate ex vivo pig eyes. The developed neural network exhibited 78.20% for the validation set and 83.89% for the test set in mean intersection over union metrics. Successful implementation and efficacy of the developed method were confirmed by comparing the preoperative and postoperative OCT volume scans from the seven experiments.

SCADE: Simultaneous Sensor Calibration and Deformation Estimation of FBG-Equipped Unmodeled Continuum Manipulators.

In this article, we present a novel stochastic algorithm called simultaneous sensor calibration and deformation estimation (SCADE) to address the problem of modeling deformation behavior of a generic continuum manipulator (CM) in free and obstructed environments. In SCADE, using a novel mathematical formulation, we introduce a priori model-independent filtering algorithm to fuse the continuous and inaccurate measurements of an embedded sensor (e.g., magnetic or piezoelectric sensors) with an intermittent but accurate data of an external imaging system (e.g., optical trackers or cameras). The main motivation of this article is the crucial need of obtaining an accurate shape/position estimation of a CM utilized in a surgical intervention. In these robotic procedures, the CM is typically equipped with an embedded sensing unit (ESU) while an external imaging modality (e.g., ultrasound or a fluoroscopy machine) is also available in the surgical site. The results of two different set of prior experiments in free and obstructed environments were used to evaluate the efficacy of SCADE algorithm. The experiments were performed with a CM specifically designed for orthopaedic interventions equipped with an inaccurate Fiber Bragg Grating (FBG) ESU and overhead camera. The results demonstrated the successful performance of the SCADE algorithm in simultaneous estimation of unknown deformation behavior of the utilized unmodeled CM together with realizing the time-varying drift of the poor-calibrated FBG sensing unit. Moreover, the results showed the phenomenal out-performance of the SCADE algorithm in estimation of the CM's tip position as compared to FBG-based position estimations.

Compact teleoperated laparoendoscopic single-site robotic surgical system: Kinematics, control, and operation.

BACKGROUND: To date a variety of teleoperated surgical robotic systems have been developed to improve a surgeon's ability to perform demanding single-port procedures. However typical large systems are bulky, expensive, and afford limited angular motion, while smaller designs suffer complications arising from limited motion range, speed, and force generation. This work was to develop and validate a simple, compact, low cost single site teleoperated laparoendoscopic surgical robotic system, with demonstrated capability to carry out basic surgical procedures. METHODS: This system builds upon previous work done at the University of Hawaii at Manoa and includes instrument and endoscope manipulators as well as compact articulated instruments designed to overcome single incision geometry complications. A robotic endoscope holder was used for the base, with an added support frame for teleoperated manipulators and instruments fabricated mostly from 3D printed parts. Kinematics and control methods were formulated for the novel manipulator configuration. RESULTS: Trajectory following results from an optical motion tracker and sample task performance results are presented. CONCLUSIONS: Results indicate that the system has successfully met the goal of basic surgical functionality while minimizing physical size, complexity, and cost.

Torque Contribution to Haptic Rendering of Virtual Textures.

Despite the fact that conventional haptic interfaces and rendering algorithms commonly approximate interactions with force only, the dynamic effects of even simple tasks, e.g., writing on a paper, involve both forces and torques. To extend previous algorithms as well as to investigate the effects of torque feedback on human roughness perception, we deployed a novel haptic platform with two probes, fingertip and penhandle. Three torque conditions were examined: 1) Slope Torque, which orients the probe perpendicular to the surface, 2) No Torque, where no active torque is provided by the device, and 3) Stiff Torque, where torque feedback is provided to keep the probe upright. A conventional magnitude estimation experiment was performed. The results indicated that both the torque signals and grasp type mediate human perception of virtual textures. Slope Torque led to greater perceived roughness when the fingertip was used, and the fingertip led to higher roughness ratings than the penhandle with the Slope Torque condition. The Slope Torque algorithm appears to be advantageous for generating rougher surfaces compared to the force-based algorithms which are typically limited by the system stability and actuator saturation.

Large Deflection Shape Sensing of a Continuum Manipulator for Minimally-Invasive Surgery.

Shape sensing techniques utilizing Fiber Bragg grating (FBG) arrays can enable real-time tracking and control of dexterous continuum manipulators (DCM) used in minimally invasive surgeries. For many surgical applications, the DCM may need to operate with much larger curvatures than what current shape sensing methods can detect. This paper proposes a novel shape sensor, which can detect a radius of curvature of 15 mm for a 35 mm long DCM. For this purpose, we used FBG sensors along with nitinol wires as the supporting substrates to form a triangular cross section. For verification, we assembled the sensor inside the wall of the DCM. Experimental results indicate that the proposed sensor can detect the DCM's curvature with an average error of 3.14%.