A usability analysis of augmented reality and haptics for surgical planning.

PURPOSE: Proper visualization and interaction with complex anatomical data can improve understanding, allowing for more intuitive surgical planning. The goal of our work was to study what the most intuitive yet practical platforms for interacting with 3D medical data are in the context of surgical planning. METHODS: We compared planning using a monitor and mouse, a monitor with a haptic device, and an augmented reality (AR) head-mounted display which uses a gesture-based interaction. To determine the most intuitive system, two user studies, one with novices and one with experts, were conducted. The studies involved planning of three scenarios: (1) heart valve repair, (2) hip tumor resection, and (3) pedicle screw placement. Task completion time, NASA Task Load Index and system-specific questionnaires were used for the evaluation. RESULTS: Both novices and experts preferred the AR system for pedicle screw placement. Novices preferred the haptic system for hip tumor planning, while experts preferred the mouse and keyboard. In the case of heart valve planning, novices preferred the AR system but there was no clear preference for experts. Both groups reported that AR provides the best spatial depth perception. CONCLUSION: The results of the user studies suggest that different surgical cases may benefit from varying interaction and visualization methods. For example, for planning surgeries with implants and instrumentations, mixed reality could provide better 3D spatial perception, whereas using landmarks for delineating specific targets may be more effective using a traditional 2D interface.

Design Optimization of a Hybrid-Driven Soft Surgical Robot with Biomimetic Constraints.

The current study investigated the geometry optimization of a hybrid-driven (based on the combination of air pressure and tendon tension) soft robot for use in robot-assisted intra-bronchial intervention. Soft robots, made from compliant materials, have gained popularity for use in surgical interventions due to their dexterity and safety. The current study aimed to design a catheter-like soft robot with an improved performance by minimizing radial expansion during inflation and increasing the force exerted on targeted tissues through geometry optimization. To do so, a finite element analysis (FEA) was employed to optimize the soft robot's geometry, considering a multi-objective goal function that incorporated factors such as chamber pressures, tendon tensions, and the cross-sectional area. To accomplish this, a cylindrical soft robot with three air chambers, three tendons, and a central working channel was considered. Then, the dimensions of the soft robot, including the length of the air chambers, the diameter of the air chambers, and the offsets of the air chambers and tendon routes, were optimized to minimize the goal function in an in-plane bending scenario. To accurately simulate the behavior of the soft robot, Ecoflex 00-50 samples were tested based on ISO 7743, and a hyperplastic model was fitted on the compression test data. The FEA simulations were performed using the response surface optimization (RSO) module in ANSYS software, which iteratively explored the design space based on defined objectives and constraints. Using RSO, 45 points of experiments were generated based on the geometrical and loading constraints. During the simulations, tendon force was applied to the tip of the soft robot, while simultaneously, air pressure was applied inside the chamber. Following the optimization of the geometry, a prototype of the soft robot with the optimized values was fabricated and tested in a phantom model, mimicking simulated surgical conditions. The decreased actuation effort and radial expansion of the soft robot resulting from the optimization process have the potential to increase the performance of the manipulator. This advancement led to improved control over the soft robot while additionally minimizing unnecessary cross-sectional expansion. The study demonstrates the effectiveness of the optimization methodology for refining the soft robot's design and highlights its potential for enhancing surgical interventions.

Hyperelastic Modeling and Validation of Hybrid-Actuated Soft Robot with Pressure-Stiffening.

Soft robots have gained popularity, especially in intraluminal applications, because their soft bodies make them safer for surgical interventions than flexures with rigid backbones. This study investigates a pressure-regulating stiffness tendon-driven soft robot and provides a continuum mechanics model for it towards using that in adaptive stiffness applications. To this end, first, a central single-chamber pneumatic and tri-tendon-driven soft robot was designed and fabricated. Afterward, the classic Cosserat's rod model was adopted and augmented with the hyperelastic material model. The model was then formulated as a boundary-value problem and was solved using the shooting method. To identify the pressure-stiffening effect, a parameter-identification problem was formulated to identify the relationship between the flexural rigidity of the soft robot and internal pressure. The flexural rigidity of the robot at various pressures was optimized to match theoretical deformation and experiments. The theoretical findings of arbitrary pressures were then compared with the experiment for validation. The internal chamber pressure was in the range of 0 to 40 kPa and the tendon tensions were in the range of 0 to 3 N. The theoretical and experimental findings were in fair agreement for tip displacement with a maximum error of 6.40% of the flexure's length.

Stiffness Adaptation of a Hybrid Soft Surgical Robot for Improved Safety in Interventional Surgery.

Minimally invasive instruments are inserted per-cutaneously and are steered toward the desired anatomy. The low stiffness of instruments is an advantage; however, once the target is reached, the instrument usually is required to transmit force to the environment. The main limitation of the constant stiffness is predetermined maneuverability and cap of force transmission. Whereas, a highly flexible device can be safely steered through the body but is not suitable for payload limit, while a highly stiff device can have relatively high loads but cannot be steered in highly tortuous trajectories. To overcome this limitation, an adaptive stiffness soft robot was proposed, and the effects of the chamber pressure on the stiffness of the soft robot were investigated. To this end, a single-chamber pneumatic soft robot with one tendon was designed and fabricated. Afterward, a continuum mechanics model based on the nonlinear Cosserat rod model with hyperelastic material model and large deformation kinematics of the robot was developed. The shooting method solved the model as a boundary value problem with Dirichlet and Neumann boundary conditions. The results of the model showed stiffness adaptation feasibility with simultaneous tendon-driving and pneumatic actuation. Thus, to validate the theoretical findings, a series of experimental studies were performed with pressure in the range of 33 to 44 kPa and tendon tensions in the range of 0 to 2.7 N. The theoretical and experimental results for tip displacement and stiffness showed similar trends with a maximum error of 8.25%.

Comparison of Mechanistic and Learning-based Tip Force Estimation on Tendon-driven Soft Robotic Catheters.

Researchers have adopted mechanistic and learning-based approaches for tip force estimation on soft robotic catheters. Typically the literature attributes the mech-anistic methods with more accuracy while indicating the learning-based methods outpace in computational time. In this study, a previously validated mechanistic tip force estimation method was compared with four learning-based methods, i.e. support-vector-regression (SVR), random-forest (RF), Ad-aBoost (Ada), and deep neural network (DNN). The learning-based methods were trained on experimental data acquired from a robotic catheter, developed in-house. The accuracy of force estimation using the five methods were compared with the ground truth forces in a teleoperated catheter manipulation test. Moreover, the capability of the learning-based models in contact detection, i.e., detection of the onset of tip contact, were compared with the ground truth. The results showed that the mechanical model had a mean-absolute error (MAE) of 8.8 mN while the MAE of SVR, RF, Ada, and DNN were 5.6, 5.2, 5.3, and 5.1 mN, respectively. Moreover, the accuracy and precision of the mechanistic model for contact detection was 89.2% and 91.7%, respectively, while these were 97.0%, 97.7%, 97.6%,and 97% and 97.9%, 98.3%, 97.8%, and 98.8% for the SVR, RF, Ada, and DNN, respectively. The comparison showed that with hyper-parameter optimization the learning-based models surpassed the mechanistic model in accuracy and precision, while both method approaches revealed acceptable performance for the proposed application.

Toward Task Autonomy in Robotic Cardiac Ablation: Learning-Based Kinematic Control of Soft Tendon-Driven Catheters.

The goal of this study was to propose and validate a control framework with level-2 autonomy (task autonomy) for the control of flexible ablation catheters. To this end, a kinematic model for the flexible portion of typical ablation catheters was developed and a 40-mm-long spring-loaded flexible catheter was fabricated. The feasible space of the catheter was obtained experimentally. Furthermore, a robotic catheter intervention system was prototyped for controlling the length of the catheter tendons. The proposed control framework used a support vector machine classifier to determine the tendons to be driven, and a fully connected neural network regressor to determine the length of the tendons. The classifier and regressors were trained with the data from the feasible space. The control system was implemented in parallel at user-interface and firmware and exhibited a 0.4-s lag in following the input. The validation studies were four trajectory tracking and four target reaching experiments. The system was capable of tracking trajectories with an error of 0.49 ± 0.32 and 0.62 ± 0.36 mm in slow and fast trajectories, respectively. Also, it exhibited submillimeter accuracy in reaching three preplanned targets and ruling out one nonfeasible target autonomously. The results showed improved accuracy and repeatability of the position control compared with the recent literature. The proposed learning-based approach could be used in enabling task autonomy for catheter-based ablation therapies.

Sensor-free Force Control of Tendon-driven Ablation Catheters through Position Control and Contact Modeling.

In the present study, a sensor-free force control framework for tendon-driven steerable catheters was proposed and validated. The hypothesis of this study was that the contact force between the catheter tip and the tissue could be controlled using the estimated force with a previously validated displacement-based viscoelastic tissue model. The tissue model was used in a feedback control loop. The model estimated the contact force based on a realtime estimation of catheter-tissue indentation depth performed by a data-driven inverse kinematic model. To test the hypothesis, a tendon-driven catheter (φ6 × 40mm) and a robotic catheter intervention system were prototyped and characterized. Three validation studies were performed to test the performance of the proposed system with static and dynamic inputs. The results showed that the system was capable of reaching to the desired force with a root-mean-square error of 0.03 ± 0.02N for static tests and 0.05 ± 0.04N for dynamic inputs. The main contribution of this study was providing a computationally efficient and sensor-free force control schema for tendon-driven catheters.

Development and assessment of a stiffness display system for minimally invasive surgery based on smart magneto-rheological elastomers.

In the present study, a solution to address the clinical need for stiffness display during manual and robotic minimally invasive surgery was postulated, developed, and assessed. To this end, a magneto-rheological elastomer-based stiffness display, MiTouch, was designed, developed, and analyzed. The mechanical properties of the MRE and system parameters were identified experimentally, based on which the force-field-stiffness response surface of the smart MRE was characterized. Based on the response surface, a stiffness controller was designed and verified for a set of performance requirements. A heartbeat simulation experiment showed the capability of the system for replicating desired tactile forces through stiffness control. Also, the system successfully attained an arbitrarily selected stiffness (4 N/mm) and maintained it within a bounded range (4.07 ± 0.41 N/mm). A comparison of the system performance with current literature validated its applicability for the proposed medical application.

Haptic Telerobotic Cardiovascular Intervention: A Review of Approaches, Methods, and Future Perspectives.

Cardiac diseases are recognized as the leading cause of mortality, hospitalization, and medical prescription globally. The gold standard for the treatment of coronary artery stenosis is the percutaneous cardiac intervention that is performed under live X-ray imaging. Substantial clinical evidence shows that the surgeon and staff are prone to serious health problems due to X-ray exposure and occupational hazards. Telerobotic vascular intervention systems with a master-slave architecture reduced the X-ray exposure and enhanced the clinical outcomes; however, the loss of haptic feedback during surgery has been the main limitation of such systems. This paper is a review of the state of the art for haptic telerobotic cardiovascular interventions. A survey on the literature published between 2000 and 2019 was performed. Results of the survey were screened based on their relevance to this paper. Also, the leading research disciplines were identified based on the results of the survey. Furthermore, different approaches for sensor-based and model-based haptic telerobotic cardiovascular intervention, haptic rendering and actuation, and the pertinent methods were critically reviewed and compared. In the end, the current limitations of the state of the art, unexplored research areas as well as the future perspective of the research on this technology were laid out.

Hybrid piezoresistive-optical tactile sensor for simultaneous measurement of tissue stiffness and detection of tissue discontinuity in robot-assisted minimally invasive surgery.

To compensate for the lack of touch during minimally invasive and robotic surgeries, tactile sensors are integrated with surgical instruments. Surgical tools with tactile sensors have been used mainly for distinguishing among different tissues and detecting malignant tissues or tumors. Studies have revealed that malignant tissue is most likely stiffer than normal. This would lead to the formation of a sharp discontinuity in tissue mechanical properties. A hybrid piezoresistive-optical-fiber sensor is proposed. This sensor is investigated for its capabilities in tissue distinction and detection of a sharp discontinuity. The dynamic interaction of the sensor and tissue is studied using finite element method. The tissue is modeled as a two-term Mooney­Rivlin hyperelastic material. For experimental verification, the sensor was microfabricated and tested under the same conditions as of the simulations. The simulation and experimental results are in a fair agreement. The sensor exhibits an acceptable linearity, repeatability, and sensitivity in characterizing the stiffness of different tissue phantoms. Also, it is capable of locating the position of a sharp discontinuity in the tissue. Due to the simplicity of its sensing principle, the proposed hybrid sensor could also be used for industrial applications.