Patient 3D body pose estimation from pressure imaging.

PURPOSE: In-bed motion monitoring has become of great interest for a variety of clinical applications. Image-based approaches could be seen as a natural non-intrusive approach for this purpose; however, video devices require special challenging settings for a clinical environment. We propose to estimate the patient's posture from pressure sensors' data mapped to images. METHODS: We introduce a deep learning method to retrieve human poses from pressure sensors data. In addition, we present a second approach that is based on a hashing content-retrieval approach. RESULTS: Our results show good performance with both presented methods even in poses where the subject has minimal contact with the sensors. Moreover, we show that deep learning approaches could be used in this medical application despite the limited amount of available training data. Our ConvNet approach provides an overall posture even when the patient has less contact with the mattress surface. In addition, we show that both methods could be used in real-time patient monitoring. CONCLUSIONS: We have provided two methods to successfully perform real-time in-bed patient pose estimation, which is robust to different sizes of patient and activities. Furthermore, it can provide an overall posture even when the patient has less contact with the mattress surface.

Robust navigation support in lowest dose image setting.

PURPOSE: Clinical cardiac electrophysiology (EP) is concerned with diagnosis and treatment of cardiac arrhythmia describing abnormality or perturbation in the normal activation sequence of the myocardium. With the recent introduction of lowest dose X-ray imaging protocol for EP procedures, interventional image enhancement has gained crucial importance for the well-being of patients as well as medical staff. METHODS: In this paper, we introduce a novel method to detect and track different EP catheter electrodes in lowest dose fluoroscopic sequences based on [Formula: see text]-sparse coding and online robust PCA (ORPCA). Besides being able to work on real lowest dose sequences, the underlying methodology achieves simultaneous detection and tracking of three main EP catheters used during ablation procedures. RESULTS: We have validated our algorithm on 16 lowest dose fluoroscopic sequences acquired during real cardiac ablation procedures. In addition to expert labels for 2 sequences, we have employed a crowdsourcing strategy to obtain ground truth labels for the remaining 14 sequences. In order to validate the effect of different training data, we have employed a leave-one-out cross-validation scheme yielding an average detection rate of [Formula: see text]. CONCLUSION: Besides these promising quantitative results, our medical partners also expressed their high satisfaction. Being based on [Formula: see text]-sparse coding and online robust PCA (ORPCA), our method advances previous approaches by being able to detect and track electrodes attached to multiple different catheters.

A photon recycling approach to the denoising of ultra-low dose X-ray sequences.

PURPOSE: Clinical procedures that make use of fluoroscopy may expose patients as well as the clinical staff (throughout their career) to non-negligible doses of radiation. The potential consequences of such exposures fall under two categories, namely stochastic (mostly cancer) and deterministic risks (skin injury). According to the "as low as reasonably achievable" principle, the radiation dose can be lowered only if the necessary image quality can be maintained. METHODS: Our work improves upon the existing patch-based denoising algorithms by utilizing a more sophisticated noise model to exploit non-local self-similarity better and this in turn improves the performance of low-rank approximation. The novelty of the proposed approach lies in its properly designed and parameterized noise model and the elimination of initial estimates. This reduces the computational cost significantly. RESULTS: The algorithm has been evaluated on 500 clinical images (7 patients, 20 sequences, 3 clinical sites), taken at ultra-low dose levels, i.e. 50% of the standard low dose level, during electrophysiology procedures. An average improvement in the contrast-to-noise ratio (CNR) by a factor of around 3.5 has been found. This is associated with an image quality achieved at around 12 (square of 3.5) times the ultra-low dose level. Qualitative evaluation by X-ray image quality experts suggests that the method produces denoised images that comply with the required image quality criteria. CONCLUSION: The results are consistent with the number of patches used, and they demonstrate that it is possible to use motion estimation techniques and "recycle" photons from previous frames to improve the image quality of the current frame. Our results are comparable in terms of CNR to Video Block Matching 3D-a state-of-the-art denoising method. But qualitative analysis by experts confirms that the denoised ultra-low dose X-ray images obtained using our method are more realistic with respect to appearance.

Vascular image registration techniques: A living review.

Registration of vascular structures is crucial for preoperative planning, intraoperative navigation, and follow-up assessment. Typical applications include, but are not limited to, Trans-catheter Aortic Valve Implantation and monitoring of tumor vasculature or aneurysm growth. In order to achieve the aforementioned goals, a large number of various registration algorithms has been developed. With this review paper we provide a comprehensive overview over the plethora of existing techniques with a particular focus on the suitable classification criteria such as the involved modalities of the employed optimization methods. However, we wish to go beyond a static literature review which is naturally doomed to be outdated after a certain period of time due to the research progress. We augment this review paper with an extendable and interactive database in order to obtain a living review whose currency goes beyond the one of a printed paper. All papers in this database are labeled with one or multiple tags according to 13 carefully defined categories. The classification of all entries can then be visualized as one or multiple trees which are presented via a web-based interactive app (http://livingreview.in.tum.de) allowing the user to choose a unique perspective for literature review. In addition, the user can search the underlying database for specific tags or publications related to vessel registration. Many applications of this framework are conceivable, including the use for getting a general overview on the topic or the utilization by physicians for deciding about the best-suited algorithm for a specific application.

Single-view X-ray depth recovery: toward a novel concept for image-guided interventions.

PURPOSE: X-ray imaging is widely used for guiding minimally invasive surgeries. Despite ongoing efforts in particular toward advanced visualization incorporating mixed reality concepts, correct depth perception from X-ray imaging is still hampered due to its projective nature. METHODS: In this paper, we introduce a new concept for predicting depth information from single-view X-ray images. Patient-specific training data for depth and corresponding X-ray attenuation information are constructed using readily available preoperative 3D image information. The corresponding depth model is learned employing a novel label-consistent dictionary learning method incorporating atlas and spatial prior constraints to allow for efficient reconstruction performance. RESULTS: We have validated our algorithm on patient data acquired for different anatomy focus (abdomen and thorax). Of 100 image pairs per each of 6 experimental instances, 80 images have been used for training and 20 for testing. Depth estimation results have been compared to ground truth depth values. CONCLUSION: We have achieved around [Formula: see text] and [Formula: see text] mean squared error on abdomen and thorax datasets, respectively, and visual results of our proposed method are very promising. We have therefore presented a new concept for enhancing depth perception for image-guided interventions.

AggNet: Deep Learning From Crowds for Mitosis Detection in Breast Cancer Histology Images.

The lack of publicly available ground-truth data has been identified as the major challenge for transferring recent developments in deep learning to the biomedical imaging domain. Though crowdsourcing has enabled annotation of large scale databases for real world images, its application for biomedical purposes requires a deeper understanding and hence, more precise definition of the actual annotation task. The fact that expert tasks are being outsourced to non-expert users may lead to noisy annotations introducing disagreement between users. Despite being a valuable resource for learning annotation models from crowdsourcing, conventional machine-learning methods may have difficulties dealing with noisy annotations during training. In this manuscript, we present a new concept for learning from crowds that handle data aggregation directly as part of the learning process of the convolutional neural network (CNN) via additional crowdsourcing layer (AggNet). Besides, we present an experimental study on learning from crowds designed to answer the following questions. 1) Can deep CNN be trained with data collected from crowdsourcing? 2) How to adapt the CNN to train on multiple types of annotation datasets (ground truth and crowd-based)? 3) How does the choice of annotation and aggregation affect the accuracy? Our experimental setup involved Annot8, a self-implemented web-platform based on Crowdflower API realizing image annotation tasks for a publicly available biomedical image database. Our results give valuable insights into the functionality of deep CNN learning from crowd annotations and prove the necessity of data aggregation integration.

Online tracking of interventional devices for endovascular aortic repair.

PURPOSE: The continuous integration of innovative imaging modalities into conventional vascular surgery rooms has led to an urgent need for computer assistance solutions that support the smooth integration of imaging within the surgical workflow. In particular, endovascular interventions performed under 2D fluoroscopic or angiographic imaging only, require reliable and fast navigation support for complex treatment procedures such as endovascular aortic repair. Despite the vast variety of image-based guide wire and catheter tracking methods, an adoption of these for detecting and tracking the stent graft delivery device is not possible due to its special geometry and intensity appearance. METHODS: In this paper, we present, for the first time, the automatic detection and tracking of the stent graft delivery device in 2D fluoroscopic sequences on the fly. The proposed approach is based on the robust principal component analysis and extends the conventional batch processing towards an online tracking system that is able to detect and track medical devices on the fly. RESULTS: The proposed method has been tested on interventional sequences of four different clinical cases. In the lack of publicly available ground truth data, we have further initiated a crowd sourcing strategy that has resulted in 200 annotations by unexperienced users, 120 of which were used to establish a ground truth dataset for quantitatively evaluating our algorithm. In addition, we have performed a user study amongst our clinical partners for qualitative evaluation of the results. CONCLUSIONS: Although we calculated an average error in the range of nine pixels, the fact that our tracking method functions on the fly and is able to detect stent grafts in all unfolding stages without fine-tuning of parameters has convinced our clinical partners and they all agreed on the very high clinical relevance of our method.

A Fourier-based approach to the angiographic assessment of flow diverter efficacy in the treatment of cerebral aneurysms.

Flow diversion is an emerging endovascular treatment option for cerebral aneurysms. Quantitative assessment of hemodynamic changes induced by flow diversion can aid clinical decision making in the treatment of cerebral aneurysms. In this article, besides summarizing past key research efforts, we propose a novel metric for the angiographic assessment of flow diverter deployments in the treatment of cerebral aneurysms. By analyzing the frequency spectra of signals derived from digital subtraction angiography (DSA) series, the metric aims to quantify the prevalence of frequency components that correspond to the patient-specific heart rate. Indicating the decoupling of aneurysms from healthy blood circulation, our proposed metric could advance clinical guidelines for treatment success prediction. The very promising results of a retrospective feasibility study on 26 DSA series warrant future efforts to study the validity of the proposed metric within a clinical setting.

WITHDRAWN: Editorial of special issue Computing and Visualisation for Intravascular Imaging.

The Publisher regrets that this article is an accidental duplication of an article that has already been published, http://dx.doi.org/10.1016/j.compmedimag.2014.01.001. The duplicate article has therefore been withdrawn.

Template-based CTA to x-ray angio rigid registration of coronary arteries in frequency domain with automatic x-ray segmentation.

PURPOSE: A key challenge for image guided coronary interventions is accurate and absolutely robust image registration bringing together preinterventional information extracted from a three-dimensional (3D) patient scan and live interventional image information. In this paper, the authors present a novel scheme for 3D to two-dimensional (2D) rigid registration of coronary arteries extracted from preoperative image scan (3D) and a single segmented intraoperative x-ray angio frame in frequency and spatial domains for real-time angiography interventions by C-arm fluoroscopy. METHODS: Most existing rigid registration approaches require a close initialization due to the abundance of local minima and high complexity of search algorithms. The authors' method eliminates this requirement by transforming the projections into translation-invariant Fourier domain for estimating the 3D pose. For 3D rotation recovery, template Digitally Reconstructed Radiographs (DRR) as candidate poses of 3D vessels of segmented computed tomography angiography are produced by rotating the camera (image intensifier) around the DICOM angle values with a specific range as in C-arm setup. The authors have compared the 3D poses of template DRRs with the segmented x-ray after equalizing the scales in three domains, namely, Fourier magnitude, Fourier phase, and Fourier polar. The best rotation pose candidate was chosen by one of the highest similarity measures returned by the methods in these domains. It has been noted in literature that frequency domain methods are robust against noise and occlusion which was also validated by the authors' results. 3D translation of the volume was then recovered by distance-map based BFGS optimization well suited to convex structure of the authors' objective function without local minima due to distance maps. A novel automatic x-ray vessel segmentation was also performed in this study. RESULTS: Final results were evaluated in 2D projection space for patient data; and with ground truth values and landmark distances for the images acquired with a solid phantom vessel. Results validate that rotation recovery in frequency domain is robust against differences in segmentations in two modalities. Distance-map translation is successful in aligning coronary trees with highest possible overlap. CONCLUSIONS: Numerical and qualitative results show that single view rigid alignment in projection space is successful. This work can be extended with multiple views to resolve depth ambiguity and with deformable registration to account for nonrigid motion in patient data.

Disocclusion-based 2D-3D registration for aortic interventions.

Occlusions introduced by medical instruments affect the accuracy and robustness of existing intensity-based medical image registration algorithms. In this paper, we present disocclusion-based 2D-3D registration handling occlusion and dissimilarity during registration. Therefore, we introduce two disocclusion techniques, Spline Interpolation and Stent-editing, and two robust similarity measures, Huber and Tukey Gradient Correlation. Our techniques are validated on synthetic and real interventional data and compared with well-known approaches. Results prove that an integration of disocclusion into the registration procedure yield higher accuracy and robustness. It is also shown that the robust measures have different effects depending on the type of occluding structure.

3D stent recovery from one X-ray projection.

In the current clinical workflow of endovascular abdominal aortic repairs (EVAR) a stent graft is inserted into the aneurysmatic aorta under 2D angiographic imaging. Due to the missing depth information in the X-ray visualization, it is highly difficult in particular for junior physicians to place the stent graft in the preoperatively defined position within the aorta. Therefore, advanced 3D visualization of stent grafts is highly required. In this paper, we present a novel algorithm to automatically match a 3D model of the stent graft to an intraoperative 2D image showing the device. By automatic preprocessing and a global-to-local registration approach, we are able to abandon user interaction and still meet the desired robustness. The complexity of our registration scheme is reduced by a semi-simultaneous optimization strategy incorporating constraints that correspond to the geometric model of the stent graft. Via experiments on synthetic, phantom, and real interventional data, we are able to show that the presented method matches the stent graft model to the 2D image data with good accuracy.

Endovascular image-guided navigation: validation of two volume-volume registration algorithms.

The limited volume covered by intraoperatively acquired CT scans makes the use of navigation systems difficult. Preoperative images cover a larger volume of interest. Hence, reliable registration of high quality preoperative to intraoperative CT will provide the necessary image information required for navigation. This study evaluates two algorithms (Siemens, CAMP) for volume-volume registration for usage during endovascular navigation. Twenty patients treated for abdominal aortic aneurysm were scanned with pre-, intra- and postoperative CT. Six data sets were excluded due to variations in image acquisition parameters and severe artifacts. Fourteen intra- and postoperative datasets were registered ten times with both algorithms, altogether 140 registrations for each program. In all data sets five specified landmarks placed by two radiologists were used to evaluate registration accuracy. The distance between the paired landmarks in the registered intra- and postoperative volumes was measured and the root mean square value calculated. Reference registrations were based on rigid body registration of the five landmarks in the intra- and postoperative volumes. Registration accuracy (mean ± SD) was for Siemens 5.05 ± 4.74 mm, for CAMP 4.02 ± 1.52 mm and for the reference registrations 2.72 ± 1.18 mm. The registration algorithms differed significantly, p < 0.001.