Spatio-temporal deep learning models for tip force estimation during needle insertion.

PURPOSE: Precise placement of needles is a challenge in a number of clinical applications such as brachytherapy or biopsy. Forces acting at the needle cause tissue deformation and needle deflection which in turn may lead to misplacement or injury. Hence, a number of approaches to estimate the forces at the needle have been proposed. Yet, integrating sensors into the needle tip is challenging and a careful calibration is required to obtain good force estimates. METHODS: We describe a fiber-optic needle tip force sensor design using a single OCT fiber for measurement. The fiber images the deformation of an epoxy layer placed below the needle tip which results in a stream of 1D depth profiles. We study different deep learning approaches to facilitate calibration between this spatio-temporal image data and the related forces. In particular, we propose a novel convGRU-CNN architecture for simultaneous spatial and temporal data processing. RESULTS: The needle can be adapted to different operating ranges by changing the stiffness of the epoxy layer. Likewise, calibration can be adapted by training the deep learning models. Our novel convGRU-CNN architecture results in the lowest mean absolute error of [Formula: see text] and a cross-correlation coefficient of 0.9997 and clearly outperforms the other methods. Ex vivo experiments in human prostate tissue demonstrate the needle's application. CONCLUSIONS: Our OCT-based fiber-optic sensor presents a viable alternative for needle tip force estimation. The results indicate that the rich spatio-temporal information included in the stream of images showing the deformation throughout the epoxy layer can be effectively used by deep learning models. Particularly, we demonstrate that the convGRU-CNN architecture performs favorably, making it a promising approach for other spatio-temporal learning problems.

Model checking for trigger loss detection during Doppler ultrasound-guided fetal cardiovascular MRI.

PURPOSE: Ultrasound (US) is the state of the art in prenatal diagnosis to depict fetal heart diseases. Cardiovascular magnetic resonance imaging (CMRI) has been proposed as a complementary diagnostic tool. Currently, only trigger-based methods allow the temporal and spatial resolutions necessary to depict the heart over time. Of these methods, only Doppler US (DUS)-based triggering is usable with higher field strengths. DUS is sensitive to motion. This may lead to signal and, ultimately, trigger loss. If too many triggers are lost, the image acquisition is stopped, resulting in a failed imaging sequence. Moreover, losing triggers may prolong image acquisition. Hence, if no actual trigger can be found, injected triggers are added to the signal based on the trigger history. METHOD: We use model checking, a technique originating from the computer science domain that formally checks if a model satisfies given requirements, to simultaneously model heart and respiratory motion and to decide whether respiration has a prominent effect on the signal. Using bounds on the physiological parameters and their variability, the method detects when changes in the signal are due to respiration. We use this to decide when to inject a trigger. RESULTS: In a real-world scenario, we can reduce the number of falsely injected triggers by 94% from more than 87% to less than 5%. On a subset of motion that would allow CMRI, the number can be further reduced to below 0.2%. In a study using simulations with a robot, we show that our method works for different types of motions, motion ranges, starting positions and heartbeat traces. CONCLUSION: While DUS is a promising approach for fetal CMRI, correct trigger injection is critical. Our model checking method can reduce the number of wrongly injected triggers substantially, providing a key prerequisite for fast and artifact free CMRI.

Online model checking for monitoring surrogate-based respiratory motion tracking in radiation therapy.

OBJECTIVE: Correlation between internal and external motion is critical for respiratory motion compensation in radiosurgery. Artifacts like coughing, sneezing or yawning or changes in the breathing pattern can lead to misalignment between beam and tumor and need to be detected to interrupt the treatment. We propose online model checking (OMC), a model-based verification approach from the field of formal methods, to verify that the breathing motion is regular and the correlation holds. We demonstrate that OMC may be more suitable for artifact detection than the prediction error. MATERIALS AND METHODS: We established a sinusoidal model to apply OMC to the verification of respiratory motion. The method was parameterized to detect deviations from typical breathing motion. We analyzed the performance on synthetic data and on clinical episodes showing large correlation error. In comparison, we considered the prediction error of different state-of-the-art methods based on least mean squares (LMS; normalized LMS, nLMS; wavelet-based multiscale autoregression, wLMS), recursive least squares (RLSpred) and support vector regression (SVRpred). RESULTS: On synthetic data, OMC outperformed wLMS by at least 30 % and SVRpred by at least 141 %, detecting 70 % of transitions. No artifacts were detected by nLMS and RLSpred. On patient data, OMC detected 23-49 % of the episodes correctly, outperforming nLMS, wLMS, RLSpred and SVRpred by up to 544, 491, 408 and 258 %, respectively. On selected episodes, OMC detected up to 94 % of all events. CONCLUSION: OMC is able to detect changes in breathing as well as artifacts which previously would have gone undetected, outperforming prediction error-based detection. Synthetic data analysis supports the assumption that prediction is very insensitive to specific changes in breathing. We suggest using OMC as an additional safety measure ensuring reliable and fast stopping of irradiation.