Autonomous navigation of catheters and guidewires in mechanical thrombectomy using inverse reinforcement learning.

PURPOSE: Autonomous navigation of catheters and guidewires can enhance endovascular surgery safety and efficacy, reducing procedure times and operator radiation exposure. Integrating tele-operated robotics could widen access to time-sensitive emergency procedures like mechanical thrombectomy (MT). Reinforcement learning (RL) shows potential in endovascular navigation, yet its application encounters challenges without a reward signal. This study explores the viability of autonomous guidewire navigation in MT vasculature using inverse reinforcement learning (IRL) to leverage expert demonstrations. METHODS: Employing the Simulation Open Framework Architecture (SOFA), this study established a simulation-based training and evaluation environment for MT navigation. We used IRL to infer reward functions from expert behaviour when navigating a guidewire and catheter. We utilized the soft actor-critic algorithm to train models with various reward functions and compared their performance in silico. RESULTS: We demonstrated feasibility of navigation using IRL. When evaluating single- versus dual-device (i.e. guidewire versus catheter and guidewire) tracking, both methods achieved high success rates of 95% and 96%, respectively. Dual tracking, however, utilized both devices mimicking an expert. A success rate of 100% and procedure time of 22.6 s were obtained when training with a reward function obtained through 'reward shaping'. This outperformed a dense reward function (96%, 24.9 s) and an IRL-derived reward function (48%, 59.2 s). CONCLUSIONS: We have contributed to the advancement of autonomous endovascular intervention navigation, particularly MT, by effectively employing IRL based on demonstrator expertise. The results underscore the potential of using reward shaping to efficiently train models, offering a promising avenue for enhancing the accessibility and precision of MT procedures. We envisage that future research can extend our methodology to diverse anatomical structures to enhance generalizability.

Artificial intelligence in the autonomous navigation of endovascular interventions: a systematic review.

Background: Autonomous navigation of catheters and guidewires in endovascular interventional surgery can decrease operation times, improve decision-making during surgery, and reduce operator radiation exposure while increasing access to treatment. Objective: To determine from recent literature, through a systematic review, the impact, challenges, and opportunities artificial intelligence (AI) has for the autonomous navigation of catheters and guidewires for endovascular interventions. Methods: PubMed and IEEEXplore databases were searched to identify reports of AI applied to autonomous navigation methods in endovascular interventional surgery. Eligibility criteria included studies investigating the use of AI in enabling the autonomous navigation of catheters/guidewires in endovascular interventions. Following Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA), articles were assessed using Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2). PROSPERO: CRD42023392259. Results: Four hundred and sixty-two studies fulfilled the search criteria, of which 14 studies were included for analysis. Reinforcement learning (RL) (9/14, 64%) and learning from expert demonstration (7/14, 50%) were used as data-driven models for autonomous navigation. These studies evaluated models on physical phantoms (10/14, 71%) and in-silico (4/14, 29%) models. Experiments within or around the blood vessels of the heart were reported by the majority of studies (10/14, 71%), while non-anatomical vessel platforms "idealized" for simple navigation were used in three studies (3/14, 21%), and the porcine liver venous system in one study. We observed that risk of bias and poor generalizability were present across studies. No procedures were performed on patients in any of the studies reviewed. Moreover, all studies were limited due to the lack of patient selection criteria, reference standards, and reproducibility, which resulted in a low level of evidence for clinical translation. Conclusion: Despite the potential benefits of AI applied to autonomous navigation of endovascular interventions, the field is in an experimental proof-of-concept stage, with a technology readiness level of 3. We highlight that reference standards with well-identified performance metrics are crucial to allow for comparisons of data-driven algorithms proposed in the years to come. Systematic review registration: identifier: CRD42023392259.

Recurrent neural networks for generalization towards the vessel geometry in autonomous endovascular guidewire navigation in the aortic arch.

PURPOSE: Endovascular intervention is the state-of-the-art treatment for common cardiovascular diseases, such as heart attack and stroke. Automation of the procedure may improve the working conditions of physicians and provide high-quality care to patients in remote areas, posing a major impact on overall treatment quality. However, this requires the adaption to individual patient anatomies, which currently poses an unsolved challenge. METHODS: This work investigates an endovascular guidewire controller architecture based on recurrent neural networks. The controller is evaluated in-silico on its ability to adapt to new vessel geometries when navigating through the aortic arch. The controller's generalization capabilities are examined by reducing the number of variations seen during training. For this purpose, an endovascular simulation environment is introduced, which allows guidewire navigation in a parametrizable aortic arch. RESULTS: The recurrent controller achieves a higher navigation success rate of 75.0% after 29,200 interventions compared to 71.6% after 156,800 interventions for a feedforward controller. Furthermore, the recurrent controller generalizes to previously unseen aortic arches and is robust towards size changes of the aortic arch. Being trained on 2048 aortic arch geometries gives the same results as being trained with full variation when evaluated on 1000 different geometries. For interpolation a gap of 30% of the scaling range and for extrapolation additional 10% of the scaling range can be navigated successfully. CONCLUSION: Adaption to new vessel geometries is essential in the navigation of endovascular instruments. Therefore, the intrinsic generalization to new vessel geometries poses an essential step towards autonomous endovascular robotics.

Learning-based autonomous vascular guidewire navigation without human demonstration in the venous system of a porcine liver.

PURPOSE: The navigation of endovascular guidewires is a dexterous task where physicians and patients can benefit from automation. Machine learning-based controllers are promising to help master this task. However, human-generated training data are scarce and resource-intensive to generate. We investigate if a neural network-based controller trained without human-generated data can learn human-like behaviors. METHODS: We trained and evaluated a neural network-based controller via deep reinforcement learning in a finite element simulation to navigate the venous system of a porcine liver without human-generated data. The behavior is compared to manual expert navigation, and real-world transferability is evaluated. RESULTS: The controller achieves a success rate of 100% in simulation. The controller applies a wiggling behavior, where the guidewire tip is continuously rotated alternately clockwise and counterclockwise like the human expert applies. In the ex vivo porcine liver, the success rate drops to 30%, because either the wrong branch is probed, or the guidewire becomes entangled. CONCLUSION: In this work, we prove that a learning-based controller is capable of learning human-like guidewire navigation behavior without human-generated data, therefore, mitigating the requirement to produce resource-intensive human-generated training data. Limitations are the restriction to one vessel geometry, the neglected safeness of navigation, and the reduced transferability to the real world.