Variational Level Set Evolution for Non-Rigid 3D Reconstruction From a Single Depth Camera.

We present a framework for real-time 3D reconstruction of non-rigidly moving surfaces captured with a single RGB-D camera. Based on the variational level set method, it warps a given truncated signed distance field (TSDF) to a target TSDF via gradient flow without explicit correspondence search. We optimize an energy that contains a data term which steers towards voxel-wise alignment. To ensure geometrically consistent reconstructions, we develop and compare different strategies, namely an approximately Killing vector field regularizer, gradient flow in Sobolev space and newly devised accelerated optimization. The underlying TSDF evolution makes our approach capable of capturing rapid motions, topological changes and interacting agents, but entails loss of data association. To recover correspondences, we propose to utilize the lowest-frequency Laplacian eigenfunctions of the TSDFs, which encode inherent deformation patterns. For moderate motions we are able to obtain implicit associations via a term that imposes voxel-wise eigenfunction alignment. This is not sufficient for larger motions, so we explicitly estimate voxel correspondences via signature matching of lower-dimensional eigenfunction embeddings. We carry out qualitative and quantitative evaluation of our geometric reconstruction fidelity and voxel correspondence accuracy, demonstrating advantages over related techniques in handling topological changes and fast motions.

The future of digital health with federated learning.

Data-driven machine learning (ML) has emerged as a promising approach for building accurate and robust statistical models from medical data, which is collected in huge volumes by modern healthcare systems. Existing medical data is not fully exploited by ML primarily because it sits in data silos and privacy concerns restrict access to this data. However, without access to sufficient data, ML will be prevented from reaching its full potential and, ultimately, from making the transition from research to clinical practice. This paper considers key factors contributing to this issue, explores how federated learning (FL) may provide a solution for the future of digital health and highlights the challenges and considerations that need to be addressed.

BACH: Grand challenge on breast cancer histology images.

Breast cancer is the most common invasive cancer in women, affecting more than 10% of women worldwide. Microscopic analysis of a biopsy remains one of the most important methods to diagnose the type of breast cancer. This requires specialized analysis by pathologists, in a task that i) is highly time- and cost-consuming and ii) often leads to nonconsensual results. The relevance and potential of automatic classification algorithms using hematoxylin-eosin stained histopathological images has already been demonstrated, but the reported results are still sub-optimal for clinical use. With the goal of advancing the state-of-the-art in automatic classification, the Grand Challenge on BreAst Cancer Histology images (BACH) was organized in conjunction with the 15th International Conference on Image Analysis and Recognition (ICIAR 2018). BACH aimed at the classification and localization of clinically relevant histopathological classes in microscopy and whole-slide images from a large annotated dataset, specifically compiled and made publicly available for the challenge. Following a positive response from the scientific community, a total of 64 submissions, out of 677 registrations, effectively entered the competition. The submitted algorithms improved the state-of-the-art in automatic classification of breast cancer with microscopy images to an accuracy of 87%. Convolutional neuronal networks were the most successful methodology in the BACH challenge. Detailed analysis of the collective results allowed the identification of remaining challenges in the field and recommendations for future developments. The BACH dataset remains publicly available as to promote further improvements to the field of automatic classification in digital pathology.

Classification of Polar Maps from Cardiac Perfusion Imaging with Graph-Convolutional Neural Networks.

Myocardial perfusion imaging is a non-invasive imaging technique commonly used for the diagnosis of Coronary Artery Disease and is based on the injection of radiopharmaceutical tracers into the blood stream. The patient's heart is imaged while at rest and under stress in order to determine its capacity to react to the imposed challenge. Assessment of imaging data is commonly performed by visual inspection of polar maps showing the tracer uptake in a compact, two-dimensional representation of the left ventricle. This article presents a method for automatic classification of polar maps based on graph convolutional neural networks. Furthermore, it evaluates how well localization techniques developed for standard convolutional neural networks can be used for the localization of pathological segments with respect to clinically relevant areas. The method is evaluated using 946 labeled datasets and compared quantitatively to three other neural-network-based methods. The proposed model achieves an agreement with the human observer on 89.3% of rest test polar maps and on 91.1% of stress test polar maps. Localization performed on a fine 17-segment division of the polar maps achieves an agreement of 83.1% with the human observer, while localization on a coarse 3-segment division based on the vessel beds of the left ventricle has an agreement of 78.8% with the human observer. Our method could thus assist the decision-making process of physicians when analyzing polar map data obtained from myocardial perfusion images.

Total Variation Regularization of Pose Signals with an Application to 3D Freehand Ultrasound.

Three-dimensional freehand imaging techniques are gaining wider adoption due to their ?exibility and cost ef?ciency. Typical examples for such a combination of a tracking system with an imaging device are freehand SPECT or freehand 3D ultrasound. However, the quality of the resulting image data is heavily dependent on the skill of the human operator and on the level of noise of the tracking data. The latter aspect can introduce blur or strong artifacts, which can signi?cantly hamper the interpretation of image data. Unfortunately, the most commonly used tracking systems to date, i.e. optical and electromagnetic, present a trade-off between invading the surgeon's workspace (due to line-of-sight requirements) and higher levels of noise and sensitivity due to the interference of surrounding metallic objects. In this work, we propose a novel approach for total variation regularization of data from tracking systems (which we term pose signals) based on a variational formulation in the manifold of Euclidean transformations. The performance of the proposed approach was evaluated using synthetic data as well as real ultrasound sweeps executed on both a Lego phantom and human anatomy, showing signi?cant improvement in terms of tracking data quality and compounded ultrasound images. Source code can be found at https://github.com/IFL-CAMP/pose_regularization.

Use the force: deformation correction in robotic 3D ultrasound.

PURPOSE: Ultrasound acquisitions are typically affected by deformations due to the pressure applied onto the contact surface. While a certain amount of pressure is necessary to ensure good acoustic coupling and visibility of the anatomy under examination, the caused deformations hinder accurate localization and geometric analysis of anatomical structures. These complications have even greater impact in case of 3D ultrasound scans as they limit the correct reconstruction of acquired volumes. METHODS: In this work, we propose a method to estimate and correct the induced deformation based solely on the tracked ultrasound images and information about the applied force. This is achieved by modeling estimated displacement fields of individual image sequences using the measured force information. By representing the computed displacement fields using a graph-based approach, we are able to recover a deformation-less 3D volume. RESULTS: Validation is performed on 30 in vivo human datasets acquired using a robotic ultrasound framework. Compared to ground truth, the presented deformation correction shows errors of [Formula: see text] for an applied force of 5 N at a penetration depth of 55 mm. CONCLUSION: The proposed technique allows for the correction of deformations induced by the transducer pressure in entire 3D ultrasound volumes. Our technique does not require biomechanical models, patient-specific assumptions or information about the tissue properties; it can be employed based on the information from readily available robotic ultrasound platforms.

Automatic Segmentation of Kidneys using Deep Learning for Total Kidney Volume Quantification in Autosomal Dominant Polycystic Kidney Disease.

Autosomal Dominant Polycystic Kidney Disease (ADPKD) is the most common inherited disorder of the kidneys. It is characterized by enlargement of the kidneys caused by progressive development of renal cysts, and thus assessment of total kidney volume (TKV) is crucial for studying disease progression in ADPKD. However, automatic segmentation of polycystic kidneys is a challenging task due to severe alteration in the morphology caused by non-uniform cyst formation and presence of adjacent liver cysts. In this study, an automated segmentation method based on deep learning has been proposed for TKV computation on computed tomography (CT) dataset of ADPKD patients exhibiting mild to moderate or severe renal insufficiency. The proposed method has been trained (n = 165) and tested (n = 79) on a wide range of TKV (321.2-14,670.7 mL) achieving an overall mean Dice Similarity Coefficient of 0.86 ± 0.07 (mean ± SD) between automated and manual segmentations from clinical experts and a mean correlation coefficient (ρ) of 0.98 (p < 0.001) for segmented kidney volume measurements in the entire test set. Our method facilitates fast and reproducible measurements of kidney volumes in agreement with manual segmentations from clinical experts.

Vascular image registration techniques: A living review.

Registration of vascular structures is crucial for preoperative planning, intraoperative navigation, and follow-up assessment. Typical applications include, but are not limited to, Trans-catheter Aortic Valve Implantation and monitoring of tumor vasculature or aneurysm growth. In order to achieve the aforementioned goals, a large number of various registration algorithms has been developed. With this review paper we provide a comprehensive overview over the plethora of existing techniques with a particular focus on the suitable classification criteria such as the involved modalities of the employed optimization methods. However, we wish to go beyond a static literature review which is naturally doomed to be outdated after a certain period of time due to the research progress. We augment this review paper with an extendable and interactive database in order to obtain a living review whose currency goes beyond the one of a printed paper. All papers in this database are labeled with one or multiple tags according to 13 carefully defined categories. The classification of all entries can then be visualized as one or multiple trees which are presented via a web-based interactive app (http://livingreview.in.tum.de) allowing the user to choose a unique perspective for literature review. In addition, the user can search the underlying database for specific tags or publications related to vessel registration. Many applications of this framework are conceivable, including the use for getting a general overview on the topic or the utilization by physicians for deciding about the best-suited algorithm for a specific application.

Combined Tensor Fitting and TV Regularization in Diffusion Tensor Imaging Based on a Riemannian Manifold Approach.

In this paper, we consider combined TV denoising and diffusion tensor fitting in DTI using the affine-invariant Riemannian metric on the space of diffusion tensors. Instead of first fitting the diffusion tensors, and then denoising them, we define a suitable TV type energy functional which incorporates the measured DWIs (using an inverse problem setup) and which measures the nearness of neighboring tensors in the manifold. To approach this functional, we propose generalized forward- backward splitting algorithms which combine an explicit and several implicit steps performed on a decomposition of the functional. We validate the performance of the derived algorithms on synthetic and real DTI data. In particular, we work on real 3D data. To our knowledge, the present paper describes the first approach to TV regularization in a combined manifold and inverse problem setup.

Structure-Preserving Color Normalization and Sparse Stain Separation for Histological Images.

Staining and scanning of tissue samples for microscopic examination is fraught with undesirable color variations arising from differences in raw materials and manufacturing techniques of stain vendors, staining protocols of labs, and color responses of digital scanners. When comparing tissue samples, color normalization and stain separation of the tissue images can be helpful for both pathologists and software. Techniques that are used for natural images fail to utilize structural properties of stained tissue samples and produce undesirable color distortions. The stain concentration cannot be negative. Tissue samples are stained with only a few stains and most tissue regions are characterized by at most one effective stain. We model these physical phenomena that define the tissue structure by first decomposing images in an unsupervised manner into stain density maps that are sparse and non-negative. For a given image, we combine its stain density maps with stain color basis of a pathologist-preferred target image, thus altering only its color while preserving its structure described by the maps. Stain density correlation with ground truth and preference by pathologists were higher for images normalized using our method when compared to other alternatives. We also propose a computationally faster extension of this technique for large whole-slide images that selects an appropriate patch sample instead of using the entire image to compute the stain color basis.

Joint Segmentation and Shape Regularization With a Generalized Forward-Backward Algorithm.

This paper presents a method for the simultaneous segmentation and regularization of a series of shapes from a corresponding sequence of images. Such series arise as time series of 2D images when considering video data, or as stacks of 2D images obtained by slicewise tomographic reconstruction. We first derive a model where the regularization of the shape signal is achieved by a total variation prior on the shape manifold. The method employs a modified Kendall shape space to facilitate explicit computations together with the concept of Sobolev gradients. For the proposed model, we derive an efficient and computationally accessible splitting scheme. Using a generalized forward-backward approach, our algorithm treats the total variation atoms of the splitting via proximal mappings, whereas the data terms are dealt with by gradient descent. The potential of the proposed method is demonstrated on various application examples dealing with 3D data. We explain how to extend the proposed combined approach to shape fields which, for instance, arise in the context of 3D+t imaging modalities, and show an application in this setup as well.

Multi-scale tubular structure detection in ultrasound imaging.

We propose a novel, physics-based method for detecting multi-scale tubular features in ultrasound images. The detector is based on a Hessian-matrix eigenvalue method, but unlike previous work, our detector is guided by an optimal model of vessel-like structures with respect to the ultrasound-image formation process. Our method provides a voxel-wise probability map, along with estimates of the radii and orientations of the detected tubes. These results can then be used for further processing, including segmentation and enhanced volume visualization. Most Hessian-based algorithms, including the well-known Frangi filter, were developed for CTA or MRA; they implicitly assume symmetry about the vessel centerline. This is not consistent with ultrasound data. We overcome this limitation by introducing a novel filter that allows multi-scale estimation both with respect to the vessel's centerline and with respect to the vessel's border. We use manually-segmented ultrasound imagery from 35 patients to show that our method is superior to standard Hessian-based methods. We evaluate the performance of the proposed methods based on the sensitivity and specificity like measures, and finally demonstrate further applicability of our method to vascular ultrasound images of the carotid artery, as well as ultrasound data for abdominal aortic aneurysms.

A quadratic energy minimization framework for signal loss estimation from arbitrarily sampled ultrasound data.

We present a flexible and general framework to iteratively solve quadratic energy problems on a non uniform grid, targeted at ultrasound imaging. Therefore, we model input samples as the nodes of an irregular directed graph, and define energies according to the application by setting weights to the edges. To solve the energy, we derive an effective optimization scheme, which avoids both the explicit computation of a linear system, as well as the compounding of the input data on a regular grid. The framework is validated in the context of 3D ultrasound signal loss estimation with the goal of providing an uncertainty estimate for each 3D data sample. Qualitative and quantitative results for 5 subjects and two target regions, namely US of the bone and the carotid artery, show the benefits of our approach, yielding continuous loss estimates.

3D velocity field and flow profile reconstruction from arbitrarily sampled Doppler ultrasound data.

With the need for adequate analysis of blood flow dynamics, different maging modalities have been developed to measure varying blood velocities over time. Due to its numerous advantages, Doppler ultrasound sonography remains one of the most widely used techniques in clinical routine, but requires additional preprocessing to recover 3D velocity information. Despite great progress in the last years, recent approaches do not jointly consider spatial and temporal variation in blood flow. In this work, we present a novel gating- and compounding-free method to simultaneously reconstruct a 3D velocity field and a temporal flow profile from arbitrarily sampled Doppler ultrasound measurements obtained from multiple directions. Based on a laminar flow assumption, a patch-wise B-spline formulation of blood velocity is coupled for the first time with a global waveform model acting as temporal regularization. We evaluated our method on three virtual phantom datasets, demonstrating robustness in terms of noise, angle between measurements and data sparsity, and applied it successfully to five real case datasets of carotid artery examination.

Shading correction for whole slide image using low rank and sparse decomposition.

Many microscopic imaging modalities suffer from the problem of intensity inhomogeneity due to uneven illumination or camera nonlinearity, known as shading artifacts. A typical example of this is the unwanted seam when stitching images to obtain a whole slide image (WSI). Elimination of shading plays an essential role for subsequent image processing such as segmentation, registration, or tracking. In this paper, we propose two new retrospective shading correction algorithms for WSI targeted to two common forms of WSI: multiple image tiles before mosaicking and an already-stitched image. Both methods leverage on recent achievements in matrix rank minimization and sparse signal recovery. We show how the classic shading problem in microscopy can be reformulated as a decomposition problem of low-rank and sparse components, which seeks an optimal separation of the foreground objects of interest and the background illumination field. Additionally, a sparse constraint is introduced in the Fourier domain to ensure the smoothness of the recovered background. Extensive qualitative and quantitative validation on both synthetic and real microscopy images demonstrates superior performance of the proposed methods in shading removal in comparison with a well-established method in ImageJ.

Predicate-Based Focus-and-Context Visualization for 3D Ultrasound.

Direct volume visualization techniques offer powerful insight into volumetric medical images and are part of the clinical routine for many applications. Up to now, however, their use is mostly limited to tomographic imaging modalities such as CT or MRI. With very few exceptions, such as fetal ultrasound, classic volume rendering using one-dimensional intensity-based transfer functions fails to yield satisfying results in case of ultrasound volumes. This is particularly due its gradient-like nature, a high amount of noise and speckle, and the fact that individual tissue types are rather characterized by a similar texture than by similar intensity values. Therefore, clinicians still prefer to look at 2D slices extracted from the ultrasound volume. In this work, we present an entirely novel approach to the classification and compositing stage of the volume rendering pipeline, specifically designed for use with ultrasonic images. We introduce point predicates as a generic formulation for integrating the evaluation of not only low-level information like local intensity or gradient, but also of high-level information, such as non-local image features or even anatomical models. Thus, we can successfully filter clinically relevant from non-relevant information. In order to effectively reduce the potentially high dimensionality of the predicate configuration space, we propose the predicate histogram as an intuitive user interface. This is augmented by a scribble technique to provide a comfortable metaphor for selecting predicates of interest. Assigning importance factors to the predicates allows for focus-and-context visualization that ensures to always show important (focus) regions of the data while maintaining as much context information as possible. Our method naturally integrates into standard ray casting algorithms and yields superior results in comparison to traditional methods in terms of visualizing a specific target anatomy in ultrasound volumes.

Disocclusion-based 2D-3D registration for aortic interventions.

Occlusions introduced by medical instruments affect the accuracy and robustness of existing intensity-based medical image registration algorithms. In this paper, we present disocclusion-based 2D-3D registration handling occlusion and dissimilarity during registration. Therefore, we introduce two disocclusion techniques, Spline Interpolation and Stent-editing, and two robust similarity measures, Huber and Tukey Gradient Correlation. Our techniques are validated on synthetic and real interventional data and compared with well-known approaches. Results prove that an integration of disocclusion into the registration procedure yield higher accuracy and robustness. It is also shown that the robust measures have different effects depending on the type of occluding structure.

3D stent recovery from one X-ray projection.

In the current clinical workflow of endovascular abdominal aortic repairs (EVAR) a stent graft is inserted into the aneurysmatic aorta under 2D angiographic imaging. Due to the missing depth information in the X-ray visualization, it is highly difficult in particular for junior physicians to place the stent graft in the preoperatively defined position within the aorta. Therefore, advanced 3D visualization of stent grafts is highly required. In this paper, we present a novel algorithm to automatically match a 3D model of the stent graft to an intraoperative 2D image showing the device. By automatic preprocessing and a global-to-local registration approach, we are able to abandon user interaction and still meet the desired robustness. The complexity of our registration scheme is reduced by a semi-simultaneous optimization strategy incorporating constraints that correspond to the geometric model of the stent graft. Via experiments on synthetic, phantom, and real interventional data, we are able to show that the presented method matches the stent graft model to the 2D image data with good accuracy.

Midbrain segmentation in transcranial 3D ultrasound for Parkinson diagnosis.

Ultrasound examination of the human brain through the temporal bone window, also called transcranial ultrasound (TC-US), is a completely non-invasive and cost-efficient technique, which has established itself for differential diagnosis of Parkinson's Disease (PD) in the past decade. The method requires spatial analysis of ultrasound hyperechogenicities produced by pathological changes within the Substantia Nigra (SN), which belongs to the basal ganglia within the midbrain. Related work on computer aided PD diagnosis shows the urgent need for an accurate and robust segmentation of the midbrain from 3D TC-US, which is an extremely difficult task due to poor image quality of TC-US. In contrast to 2D segmentations within earlier approaches, we develop the first method for semi-automatic midbrain segmentation from 3D TC-US and demonstrate its potential benefit on a database of 11 diagnosed Parkinson patients and 11 healthy controls.

Generalization of deformable registration in Riemannian Sobolev spaces.

In this work we discuss the generalized treatment of the deformable registration problem in Sobolev spaces. We extend previous approaches in two points: 1) by employing a general energy model which includes a regularization term, and 2) by changing the notion of distance in the Sobolev space by problem-dependent Riemannian metrics. The actual choice of the metric is such that it has a preconditioning effect on the problem, it is applicable to arbitrary similarity measures, and features a simple implementation. The experiments demonstrate an improvement in convergence and runtime by several orders of magnitude in comparison to semi-implicit gradient flows in L2. This translates to increased accuracy in practical scenarios. Furthermore, the proposed generalization establishes a theoretical link between gradient flow in Sobolev spaces and elastic registration methods.