Cascaded learning in intravascular ultrasound: coronary stent delineation in manual pullbacks.

Purpose: Implanting stents to re-open stenotic lesions during percutaneous coronary interventions is considered a standard treatment for acute or chronic coronary syndrome. Intravascular ultrasound (IVUS) can be used to guide and assess the technical success of these interventions. Automatically segmenting stent struts in IVUS sequences improves workflow efficiency but is non-trivial due to a challenging image appearance entailing manifold ambiguities with other structures. Manual, ungated IVUS pullbacks constitute a challenge in this context. We propose a fully data-driven strategy to first longitudinally detect and subsequently segment stent struts in IVUS frames. Approach: A cascaded deep learning approach is presented. It first trains an encoder model to classify frames as "stent," "no stent," or "no use." A segmentation model then delineates stent struts on a pixel level only in frames with a stent label. The first stage of the cascade acts as a gateway to reduce the risk for false positives in the second stage, the segmentation, which is trained on a smaller and difficult-to-annotate dataset. Training of the classification and segmentation model was based on 49,888 and 1826 frames of 74 sequences from 35 patients, respectively. Results: The longitudinal classification yielded Dice scores of 92.96%, 82.35%, and 94.03% for the classes stent, no stent, and no use, respectively. The segmentation achieved a Dice score of 65.1% on the stent ground truth (intra-observer performance: 75.5%) and 43.5% on all frames (including frames without stent, with guidewires, calcium, or without clinical use). The latter improved to 49.5% when gating the frames by the classification decision and further increased to 57.4% with a heuristic on the plausible stent strut area. Conclusions: A data-driven strategy for segmenting stents in ungated, manual pullbacks was presented-the most common and practical scenario in the time-critical clinical workflow. We demonstrated a mitigated risk for ambiguities and false positive predictions.

Enhanced Optical Head Tracking for Cranial Radiation Therapy: Supporting Surface Registration by Cutaneous Structures.

PURPOSE: To support surface registration in cranial radiation therapy by structural information. The risk for spatial ambiguities is minimized by using tissue thickness variations predicted from backscattered near-infrared (NIR) light from the forehead. METHODS AND MATERIALS: In a pilot study we recorded NIR surface scans by laser triangulation from 30 volunteers of different skin type. A ground truth for the soft-tissue thickness was segmented from MR scans. After initially matching the NIR scans to the MR reference, Gaussian processes were trained to predict tissue thicknesses from NIR backscatter. Moreover, motion starting from this initial registration was simulated by 5000 random transformations of the NIR scan away from the MR reference. Re-registration to the MR scan was compared with and without tissue thickness support. RESULTS: By adding prior knowledge to the backscatter features, such as incident angle and neighborhood information in the scanning grid, we showed that tissue thickness can be predicted with mean errors of <0.2 mm, irrespective of the skin type. With this additional information, the average registration error improved from 3.4 mm to 0.48 mm by a factor of 7. Misalignments of more than 1 mm were almost thoroughly (98.9%) pushed below 1 mm. CONCLUSIONS: For almost all cases tissue-enhanced matching achieved better results than purely spatial registration. Ambiguities can be minimized if the cutaneous structures do not agree. This valuable support for surface registration increases tracking robustness and avoids misalignment of tumor targets far from the registration site.

Enriching 3D optical surface scans with prior knowledge: tissue thickness computation by exploiting local neighborhoods.

PURPOSE: Patient immobilization and X-ray-based imaging provide neither a convenient nor a very accurate way to ensure low repositioning errors or to compensate for motion in cranial radiotherapy. We therefore propose an optical tracking device that exploits subcutaneous structures as landmarks in addition to merely spatial registration. To develop such head tracking algorithms, precise and robust computation of these structures is necessary. Here, we show that the tissue thickness can be predicted with high accuracy and moreover exploit local neighborhood information within the laser spot grid on the forehead to further increase this estimation accuracy. METHODS: We use statistical learning with Support Vector Regression and Gaussian Processes to learn a relationship between optical backscatter features and an MR tissue thickness ground truth. We compare different kernel functions for the data of five different subjects. The incident angle of the laser on the forehead as well as local neighborhoods is incorporated into the feature space. The latter represent the backscatter features from four neighboring laser spots. RESULTS: We confirm that the incident angle has a positive effect on the estimation error of the tissue thickness. The root-mean-square error falls even below 0.15 mm when adding the complete neighborhood information. This prior knowledge also leads to a smoothing effect on the reconstructed skin patch. Learning between different head poses yields similar results. The partial overlap of the point clouds makes the trade-off between novel information and increased feature space dimension obvious and hence feature selection by e.g., sequential forward selection necessary.

An Approach to Improve Accuracy of Optical Tracking Systems in Cranial Radiation Therapy.

This work presents a new method for the accurate estimation of soft tissue thickness based on near infrared (NIR) laser measurements. By using this estimation, our goal is to develop an improved non-invasive marker-less optical tracking system for cranial radiation therapy. Results are presented for three subjects and reveal an RMS error of less than 0.34 mm.

Efficient estimation of tissue thicknesses using sparse approximation for Gaussian processes.

Highly accurate localization of the human skull is vital in cranial radiotherapy. Marker-less optical head tracking provides a fast and accurate way to monitor this motion. Recent research has given evidence that marker-less tracking of the forehead benefits from tissue thickness information in addition to the 3D surface geometry. Using Gaussian Processes (GPs) tissue thickness is determined from optical backscatter of a sweeping laser. However, the computational complexity of the GPs scales cubically with the number of training samples. A full head scan contains 1024 points, whereas scans from several perspectives may be required for a comprehensive model for each subject. In five subjects, we thus evaluate sparse approximation methods to reduce the computational effort. We found a better - computation time versus root mean square error (RMSE) - tradeoff for a simple subset of data (SoD) technique. The increase of RMSE when dropping data was not found steep enough to justify the computational overhead of a better approximation by inducing point methods (namely FITC). Promising results were, however, obtained when clustering the training data before selecting the subset.

Controlling motion prediction errors in radiotherapy with relevance vector machines.

PURPOSE: Robotic radiotherapy can precisely ablate moving tumors when time latencies have been compensated. Recently, relevance vector machines (RVM), a probabilistic regression technique, outperformed six other prediction algorithms for respiratory compensation. The method has the distinct advantage that each predicted point is assumed to be drawn from a normal distribution. Second-order statistics, the predicted variance, were used to control RVM prediction error during a treatment and to construct hybrid prediction algorithms. METHODS: First, the duty cycle and the precision were correlated to the variance by interrupting the treatment if the variance exceeds a threshold. Second, two hybrid algorithms based on the variance were developed, one consisting of multiple RVMs (HYB(RVM)) and the other of a combination between a wavelet-based least mean square algorithm (wLMS) and a RVM (HYB(wLMS-RVM)). The variance for different motion traces was analyzed to reveal a characteristic variance pattern which gives insight in what kind of prediction errors can be controlled by the variance. RESULTS: Limiting the variance by a threshold resulted in an increased precision with a decreased duty cycle. All hybrid algorithms showed an increased prediction accuracy compared to using only their individual algorithms. The best hybrid algorithm, HYB(RVM), can decrease the mean RMSE over all 304 motion traces from 0.18 mm for a linear RVM to 0.17 mm. CONCLUSIONS: The predicted variance was shown to be an efficient metric to control prediction errors, resulting in a more robust radiotherapy treatment. The hybrid algorithm HYB(RVM) could be translated to clinical practice. It does not require further parameters, can be completely parallelised and easily further extended.

Accuracy analysis for triangulation and tracking based on time-multiplexed structured light.

PURPOSE: The authors' research group is currently developing a new optical head tracking system for intracranial radiosurgery. This tracking system utilizes infrared laser light to measure features of the soft tissue on the patient's forehead. These features are intended to offer highly accurate registration with respect to the rigid skull structure by means of compensating for the soft tissue. In this context, the system also has to be able to quickly generate accurate reconstructions of the skin surface. For this purpose, the authors have developed a laser scanning device which uses time-multiplexed structured light to triangulate surface points. METHODS: The accuracy of the authors' laser scanning device is analyzed and compared for different triangulation methods. These methods are given by the Linear-Eigen method and a nonlinear least squares method. Since Microsoft's Kinect camera represents an alternative for fast surface reconstruction, the authors' results are also compared to the triangulation accuracy of the Kinect device. Moreover, the authors' laser scanning device was used for tracking of a rigid object to determine how this process is influenced by the remaining triangulation errors. For this experiment, the scanning device was mounted to the end-effector of a robot to be able to calculate a ground truth for the tracking. RESULTS: The analysis of the triangulation accuracy of the authors' laser scanning device revealed a root mean square (RMS) error of 0.16 mm. In comparison, the analysis of the triangulation accuracy of the Kinect device revealed a RMS error of 0.89 mm. It turned out that the remaining triangulation errors only cause small inaccuracies for the tracking of a rigid object. Here, the tracking accuracy was given by a RMS translational error of 0.33 mm and a RMS rotational error of 0.12°. CONCLUSIONS: This paper shows that time-multiplexed structured light can be used to generate highly accurate reconstructions of surfaces. Furthermore, the reconstructed point sets can be used for high-accuracy tracking of objects, meeting the strict requirements of intracranial radiosurgery.

Tissue thickness estimation for high precision head-tracking using a galvanometric laser scanner - a case study.

Marker-less optical head-tracking constitutes a comfortable alternative with no exposure to radiation for realtime monitoring in radiation therapy. Supporting information such as tissue thickness has the potential to improve spatial tracking accuracy. Here we study how accurate tissue thickness can be estimated from the near-infrared (NIR) backscatter obtained from laser scans. In a case study, optical data was recorded with a galvanometric laser scanner from three subjects. A tissue ground truth from MRI was robustly matched via customized bite blocks. We show that Gaussian Processes accurately model the relationship between NIR features and tissue thickness. They were able to predict the tissue thickness with less than 0.5 mm root mean square error. Individual scaling factors for all features and an additional incident angle feature had positive effects on this performance.

Evaluation of the potential of multi-modal sensors for respiratory motion prediction and correlation.

In modern robotic radiotherapy, precise radiation of moving tumors is possible by tracking external optical surrogates. The surrogates are used to compensate for time delays and to predict internal landmarks using a correlation model. The correlation depends significantly on the surrogate position and breathing characteristics of the patient. In this context, we aim to increase the accuracy and robustness of prediction and correlation models by using a multi-modal sensor setup. Here, we evaluate the correlation coefficient of a strain belt, an acceleration and temperature sensor (air flow) with respect to external optical sensors and one internal landmark in the liver, measured by 3D ultrasound. The focus of this study is the influence of breathing artefacts, like coughing and harrumphing. Evaluating seven subjects, we found a strong decrease of the correlation for all modalities in case of artefacts. The results indicate that no precise motion compensation during these times is possible. Overall, we found that apart from the optical markers, the strain belt and temperature sensor data show the best correlation to external and internal motion.

Hidden Markov model and support vector machine based decoding of finger movements using electrocorticography.

OBJECTIVE: Support vector machines (SVM) have developed into a gold standard for accurate classification in brain-computer interfaces (BCI). The choice of the most appropriate classifier for a particular application depends on several characteristics in addition to decoding accuracy. Here we investigate the implementation of hidden Markov models (HMM) for online BCIs and discuss strategies to improve their performance. APPROACH: We compare the SVM, serving as a reference, and HMMs for classifying discrete finger movements obtained from electrocorticograms of four subjects performing a finger tapping experiment. The classifier decisions are based on a subset of low-frequency time domain and high gamma oscillation features. MAIN RESULTS: We show that decoding optimization between the two approaches is due to the way features are extracted and selected and less dependent on the classifier. An additional gain in HMM performance of up to 6% was obtained by introducing model constraints. Comparable accuracies of up to 90% were achieved with both SVM and HMM with the high gamma cortical response providing the most important decoding information for both techniques. SIGNIFICANCE: We discuss technical HMM characteristics and adaptations in the context of the presented data as well as for general BCI applications. Our findings suggest that HMMs and their characteristics are promising for efficient online BCIs.

Estimating soft tissue thickness from light-tissue interactions--a simulation study.

Immobilization and marker-based motion tracking in radiation therapy often cause decreased patient comfort. However, the more comfortable alternative of optical surface tracking is highly inaccurate due to missing point-to-point correspondences between subsequent point clouds as well as elastic deformation of soft tissue. In this study, we present a proof of concept for measuring subcutaneous features with a laser scanner setup focusing on the skin thickness as additional input for high accuracy optical surface tracking. Using Monte-Carlo simulations for multi-layered tissue, we show that informative features can be extracted from the simulated tissue reflection by integrating intensities within concentric ROIs around the laser spot center. Training a regression model with a simulated data set identifies patterns that allow for predicting skin thickness with a root mean square error of down to 18 µm. Different approaches to compensate for varying observation angles were shown to yield errors still below 90 µm. Finally, this initial study provides a very promising proof of concept and encourages research towards a practical prototype.

Optimized order estimation for autoregressive models to predict respiratory motion.

PURPOSE: To successfully ablate moving tumors in robotic radio-surgery, it is necessary to compensate for motion of inner organs caused by respiration. This can be achieved by tracking the body surface and correlating the external movement with the tumor position as it is implemented in the CyberKnife[Formula: see text] Synchrony system. Tracking errors, originating from system immanent time delays, are typically reduced by time series prediction. Many prediction algorithms exploit autoregressive (AR) properties of the signal. Estimating the optimal model order [Formula: see text] for these algorithms constitutes a challenge often solved via grid search or prior knowledge about the signal. METHODS: Aiming at a more efficient approach instead, this study evaluates the Akaike information criterion (AIC), the corrected AIC, and the Bayesian information criterion (BIC) on the first minute of the respiratory signal. Exemplarily, we evaluated the approach for a least mean square (LMS) and a wavelet-based LMS (wLMS) predictor. RESULTS: Analyzing 12 motion traces, orders estimated by AIC had the highest prediction accuracy for both prediction algorithms. Extending the investigations to 304 real motion traces, the prediction error of wLMS using AIC was found to decrease significantly by 85.1 % of the data compared to the original implementation CONCLUSIONS: The overall results suggest that using AIC to estimate the model order [Formula: see text] for prediction algorithms based on AR properties is a valid method which avoids intensive grid search and leads to high prediction accuracy.

Respiratory motion compensation with relevance vector machines.

In modern robotic radiation therapy, tumor movements due to respiration can be compensated. The accuracy of these methods can be increased by time series prediction of external optical surrogates. An algorithm based on relevance vector machines (RVM) is introduced. We evaluate RVM with linear and nonlinear basis functions on a real patient data set containing 304 motion traces and compare it with a wavelet based least mean square algorithm (wLMS), the best algorithm for this data set so far. Linear RVM outperforms wLMS significantly and increases the prediction accuracy for 80.3% of the data. We show that real time prediction is possible in case of linear RVM and discuss how the predicted variance can be used to construct promising hybrid algorithms, which further reduce the prediction error.