Using Deep Neural Networks to Improve Contact Wrench Estimation of Serial Robotic Manipulators in Static Tasks.

Reliable force-driven robot-interaction requires precise contact wrench measurements. In most robot systems these measurements are severely incorrect and in most manipulation tasks expensive additional force sensors are installed. We follow a learning approach to train the dependencies between joint torques and end-effector contact wrenches. We used a redundant serial light-weight manipulator (KUKA iiwa 7 R800) with integrated force estimation based on the joint torques measured in each of the robot's seven axes. Firstly, a simulated dataset is created to let a feed-forward net learn the relationship between end-effector contact wrenches and joint torques for a static case. Secondly, an extensive real training dataset was acquired with 330,000 randomized robot positions and end-effector contact wrenches and used for retraining the simulated trained feed-forward net. We can show that the wrench prediction error could be reduced by around 57% for the forces compared to the manufacturer's proprietary force estimation model. In addition, we show that the number of high outliers can be reduced substantially. Furthermore we prove that the approach could be also transferred to another robot (KUKA iiwa 14 R820) with reasonable prediction accuracy and without the need of acquiring new robot specific data.

Towards automated ultrasound imaging-robotic image acquisition in liver and prostate for long-term motion monitoring.

Real-time volumetric (4D) ultrasound has shown high potential for diagnostic and therapy guidance tasks. One of the main drawbacks of ultrasound imaging to date is the reliance on manual probe positioning and the resulting user dependence. Robotic assistance could help overcome this issue and facilitate the acquisition of long-term image data to observe dynamic processesin vivoover time. The aim of this study is to assess the feasibility of robotic probe manipulation and organ motion quantification during extended imaging sessions. The system consists of a collaborative robot and a 4D ultrasound system providing real-time data access. Five healthy volunteers received liver and prostate scans during free breathing over 30 min. Initial probe placement was performed with real-time remote control with a predefined contact force of 10 N. During scan acquisition, the probe position was continuously adjusted to the body surface motion using impedance control. Ultrasound volumes, the pose of the end-effector and the estimated contact forces were recorded. For motion analysis, one anatomical landmark was manually annotated in a subset of ultrasound frames for each experiment. Probe contact was uninterrupted over the entire scan duration in all ten sessions. Organ drift and imaging artefacts were successfully compensated using remote control. The median contact force along the probe's longitudinal axis was 10.0 N with maximum values of 13.2 and 21.3 N for liver and prostate, respectively. Forces exceeding 11 N only occurred in 0.3% of the time. Probe and landmark motion were more pronounced in the liver, with median interquartile ranges of 1.5 and 9.6 mm, compared to 0.6 and 2.7 mm in the prostate. The results show that robotic ultrasound imaging with dynamic force control can be used for stable, long-term imaging of anatomical regions affected by motion. The system facilitates the acquisition of 4D image datain vivoover extended scanning periods for the first time and holds the potential to be used for motion monitoring for therapy guidance as well as diagnostic tasks.

Boosting the effect of reward on cognitive control using TMS over the left IFJ.

Although an enhancing effect of reward on cognitive performance has been observed consistently, its neural underpinnings remain elusive. Recent evidence suggests that the inferior frontal junction (IFJ) may be a key player underlying such an enhancement by integrating motivational processes and cognitive control. However, its exact role and in particular a potential causality of IFJ activation is still unclear. In the present study, we therefore investigated the causal contributions of the left IFJ in motivated task switching by temporarily disrupting its activity using continuous theta burst stimulation (cTBS, Exp.1) or 1 Hz repetitive transcranial magnetic stimulation (rTMS, Exp.2). After TMS application over the left IFJ or a control site (vertex), participants performed a switch task in which numbers had to be judged by magnitude or parity. Different amounts of monetary rewards (high vs low) were used to manipulate the participants' motivational states. We measured reaction times and error rates. Irrespective of TMS stimulation, participants exhibited slower responses following task switches compared to task repeats. This effect was reduced in high reward trials. Importantly, we found that disrupting the IFJ improved participants' behavioral performance in the high reward condition. For high reward trials exclusively, error rates decreased when the IFJ was modulated with cTBS or 1 Hz rTMS but not after vertex stimulation. Our results suggest that the left IFJ is causally related to the increase in cognitive performance through reward.

Efficient Registration of High-Resolution Feature Enhanced Point Clouds.

We present a novel framework for rigid point cloud registration. Our approach is based on the principles of mechanics and thermodynamics. We solve the registration problem by assuming point clouds as rigid bodies consisting of particles. Forces can be applied between both particle systems so that they attract or repel each other. These forces are used to cause rigid-body motion of one particle system toward the other, until both are aligned. The framework supports physics-based registration processes with arbitrary driving forces, depending on the desired behaviour. Additionally, the approach handles feature-enhanced point clouds, e.g., by colours or intensity values. Our framework is freely accessible for download. In contrast to already existing algorithms, our contribution is to precisely register high-resolution point clouds with nearly constant computational effort and without the need for pre-processing, sub-sampling or pre-alignment. At the same time, the quality is up to 28 percent higher than for state-of-the-art algorithms and up to 49 percent higher when considering feature-enhanced point clouds. Even in the presence of noise, our registration approach is one of the most robust, on par with state-of-the-art implementations.

A visual probe positioning tool for 4D ultrasound-guided radiotherapy.

Ultrasound (US) guidance is a rapidly growing area in image-guided radiotherapy. For motion compensation, the therapy target needs to be visualized with the US probe to continuously determine its position and adapt for shifts. While US has obvious benefits such as real-time capability and proven safety, one of the main drawbacks to date is its user dependency - high quality results require long years of clinical experience. To provide positioning assistance for the setup of US equipment by non-experts, we developed a visual guidance tool combining real-time US volume and CT visualization in a geometrically calibrated setup. By using a 4D US station with real-time data access and an optical tracking system, we achieved a calibration accuracy of 1.2 mm and a mean 2D contour distance of 1.7 mm between organ boundaries identified in US and CT. With this low calibration error as well as the good visual alignment of the structures, the developed probe positioning tool could be a valuable aid for ultrasound-guided radiotherapy and other interventions by guiding the user to a suitable acoustic window while potentially improving setup reproducibility.

Navigation and visualisation with HoloLens in endovascular aortic repair.

INTRODUCTION: Endovascular aortic repair (EVAR) is a minimal-invasive technique that prevents life-threatening rupture in patients with aortic pathologies by implantation of an endoluminal stent graft. During the endovascular procedure, device navigation is currently performed by fluoroscopy in combination with digital subtraction angiography. This study presents the current iterative process of biomedical engineering within the disruptive interdisciplinary project Nav EVAR, which includes advanced navigation, image techniques and augmented reality with the aim of reducing side effects (namely radiation exposure and contrast agent administration) and optimising visualisation during EVAR procedures. This article describes the current prototype developed in this project and the experiments conducted to evaluate it. METHODS: The current approach of the Nav EVAR project is guiding EVAR interventions in real-time with an electromagnetic tracking system after attaching a sensor on the catheter tip and displaying this information on Microsoft HoloLens glasses. This augmented reality technology enables the visualisation of virtual objects superimposed on the real environment. These virtual objects include three-dimensional (3D) objects (namely 3D models of the skin and vascular structures) and two-dimensional (2D) objects [namely orthogonal views of computed tomography (CT) angiograms, 2D images of 3D vascular models, and 2D images of a new virtual angioscopy whose appearance of the vessel wall follows that shown in ex vivo and in vivo angioscopies]. Specific external markers were designed to be used as landmarks in the registration process to map the tracking data and radiological data into a common space. In addition, the use of real-time 3D ultrasound (US) is also under evaluation in the Nav EVAR project for guiding endovascular tools and updating navigation with intraoperative imaging. US volumes are streamed from the US system to HoloLens and visualised at a certain distance from the probe by tracking augmented reality markers. A human model torso that includes a 3D printed patient-specific aortic model was built to provide a realistic test environment for evaluation of technical components in the Nav EVAR project. The solutions presented in this study were tested by using an US training model and the aortic-aneurysm phantom. RESULTS: During the navigation of the catheter tip in the US training model, the 3D models of the phantom surface and vessels were visualised on HoloLens. In addition, a virtual angioscopy was also built from a CT scan of the aortic-aneurysm phantom. The external markers designed for this study were visible in the CT scan and the electromagnetically tracked pointer fitted in each marker hole. US volumes of the US training model were sent from the US system to HoloLens in order to display them, showing a latency of 259±86 ms (mean±standard deviation). CONCLUSION: The Nav EVAR project tackles the problem of radiation exposure and contrast agent administration during EVAR interventions by using a multidisciplinary approach to guide the endovascular tools. Its current state presents several limitations such as the rigid alignment between preoperative data and the simulated patient. Nevertheless, the techniques shown in this study in combination with fibre Bragg gratings and optical coherence tomography are a promising approach to overcome the problems of EVAR interventions.

Towards X-ray free endovascular interventions - using HoloLens for on-line holographic visualisation.

A major challenge during endovascular interventions is visualising the position and orientation of the catheter being inserted. This is typically achieved by intermittent X-ray imaging. Since the radiation exposure to the surgeon is considerable, it is desirable to reduce X-ray exposure to the bare minimum needed. Additionally, transferring two-dimensional (2D) X-ray images to 3D locations is challenging. The authors present the development of a real-time navigation framework, which allows a 3D holographic view of the vascular system without any need of radiation. They extract the patient's surface and vascular tree from pre-operative computed tomography data and register it to the patient using a magnetic tracking system. The system was evaluated on an anthropomorphic full-body phantom by experienced clinicians using a four-point questionnaire. The average score of the system (maximum of 20) was found to be 17.5. The authors' approach shows great potential to improve the workflow for endovascular procedures, by simultaneously reducing X-ray exposure. It will also improve the learning curve and help novices to more quickly master the required skills.

Impact of robotic ultrasound image guidance on plan quality in SBRT of the prostate.

OBJECTIVE: Ultrasound provides good image quality, fast volumetric imaging and is established for abdominal image guidance. Robotic transducer placement may facilitate intrafractional motion compensation in radiation therapy. We consider integration with the CyberKnife and study whether the kinematic redundancy of a seven-degrees-of-freedom robot allows for acceptable plan quality for prostate treatments. METHODS: Reference treatment plans were generated for 10 prostate cancer cases previously treated with the CyberKnife. Considering transducer and prostate motion by different safety margins, 10 different robot poses, and 3 different elbow configurations, we removed all beams colliding with robot or transducer. For each combination, plans were generated using the same strict dose constraints and the objective to maximize the target coverage. Additionally, plans for the union of all unblocked beams were generated. RESULTS: In 9 cases the planning target coverage with the ultrasound robot was within 1.1 percentage points of the reference coverage. It was 1.7 percentage points for one large prostate. For one preferable robot position, kinematic redundancy decreased the average number of blocked beam directions from 23.1 to 14.5. CONCLUSION: The impact of beam blocking can largely be offset by treatment planning and using a kinematically redundant robot. Plan quality can be maintained by carefully choosing the ultrasound robot position and pose. For smaller planning target volumes the difference in coverage is negligible for safety margins of up to 35 mm. Advances in knowledge: Integrating a robot for online intrafractional image guidance based on ultrasound can be realized while maintaining acceptable plan quality for prostate cancer treatments with the CyberKnife.

Robotic ultrasound-guided SBRT of the prostate: feasibility with respect to plan quality.

PURPOSE: Advances in radiation therapy delivery systems have enabled motion compensated SBRT of the prostate. A remaining challenge is the integration of fast, non-ionizing volumetric imaging. Recently, robotic ultrasound has been proposed as an intra-fraction image modality. We study the impact of integrating a light-weight robotic arm carrying an ultrasound probe with the CyberKnife system. Particularly, we analyze the effect of different robot poses on the plan quality. METHODS: A method to detect the collision of beams with the robot or the transducer was developed and integrated into our treatment planning system. A safety margin accounts for beam motion and uncertainties. Using strict dose bounds and the objective to maximize target coverage, we generated a total of 7650 treatment plans for five different prostate cases. For each case, ten different poses of the ultrasound robot and transducer were considered. The effect of different sets of beam source positions and different motion margins ranging from 5 to 50 mm was analyzed. RESULTS: Compared to reference plans without the ultrasound robot, the coverage typically drops for all poses. Depending on the patient, the robot pose, and the motion margin, the reduction in coverage may be up to 50 % points. However, for all patient cases, there exist poses for which the loss in coverage was below 1 % point for motion margins of up to 20 mm. In general, there is a positive correlation between the number of treatment beams and the coverage. CONCLUSION: While the blocking of beam directions has a negative effect on the plan quality, the results indicate that a careful choice of the ultrasound robot's pose and a large solid angle covered by beam starting positions can offset this effect. Identifying robot poses that yield acceptable plan quality and allow for intra-fraction ultrasound image guidance, therefore, appears feasible.