Deep learning ensembles for detecting brain metastases in longitudinal multi-modal MRI studies.

Metastatic brain cancer is a condition characterized by the migration of cancer cells to the brain from extracranial sites. Notably, metastatic brain tumors surpass primary brain tumors in prevalence by a significant factor, they exhibit an aggressive growth potential and have the capacity to spread across diverse cerebral locations simultaneously. Magnetic resonance imaging (MRI) scans of individuals afflicted with metastatic brain tumors unveil a wide spectrum of characteristics. These lesions vary in size and quantity, spanning from tiny nodules to substantial masses captured within MRI. Patients may present with a limited number of lesions or an extensive burden of hundreds of them. Moreover, longitudinal studies may depict surgical resection cavities, as well as areas of necrosis or edema. Thus, the manual analysis of such MRI scans is difficult, user-dependent and cost-inefficient, and - importantly - it lacks reproducibility. We address these challenges and propose a pipeline for detecting and analyzing brain metastases in longitudinal studies, which benefits from an ensemble of various deep learning architectures originally designed for different downstream tasks (detection and segmentation). The experiments, performed over 275 multi-modal MRI scans of 87 patients acquired in 53 sites, coupled with rigorously validated manual annotations, revealed that our pipeline, built upon open-source tools to ensure its reproducibility, offers high-quality detection, and allows for precisely tracking the disease progression. To objectively quantify the generalizability of models, we introduce a new data stratification approach that accommodates the heterogeneity of the dataset and is used to elaborate training-test splits in a data-robust manner, alongside a new set of quality metrics to objectively assess algorithms. Our system provides a fully automatic and quantitative approach that may support physicians in a laborious process of disease progression tracking and evaluation of treatment efficacy.

Detecting liver cirrhosis in computed tomography scans using clinically-inspired and radiomic features.

Hepatic cirrhosis is an increasing cause of mortality in developed countries-it is the pathological sequela of chronic liver diseases, and the final liver fibrosis stage. Since cirrhosis evolves from the asymptomatic phase, it is of paramount importance to detect it as quickly as possible, because entering the symptomatic phase commonly leads to hospitalization and can be fatal. Understanding the state of the liver based on the abdominal computed tomography (CT) scans is tedious, user-dependent and lacks reproducibility. We tackle these issues and propose an end-to-end and reproducible approach for detecting cirrhosis from CT. It benefits from the introduced clinically-inspired features that reflect the patient's characteristics which are often investigated by experienced radiologists during the screening process. Such features are coupled with the radiomic ones extracted from the liver, and from the suggested region of interest which captures the liver's boundary. The rigorous experiments, performed over two heterogeneous clinical datasets (two cohorts of 241 and 32 patients) revealed that extracting radiomic features from the liver's rectified contour is pivotal to enhance the classification abilities of the supervised learners. Also, capturing clinically-inspired image features significantly improved the performance of such models, and the proposed features were consistently selected as the important ones. Finally, we showed that selecting the most discriminative features leads to the Pareto-optimal models with enhanced feature-level interpretability, as the number of features was dramatically reduced (280×) from thousands to tens.

Training and deploying a deep learning model for endoscopic severity grading in ulcerative colitis using multicenter clinical trial data.

INTRODUCTION: The Mayo Clinic Endoscopic Subscore is a commonly used grading system to assess the severity of ulcerative colitis. Correctly grading colonoscopies using the Mayo Clinic Endoscopic Subscore is a challenging task, with suboptimal rates of interrater and intrarater variability observed even among experienced and sufficiently trained experts. In recent years, several machine learning algorithms have been proposed in an effort to improve the standardization and reproducibility of Mayo Clinic Endoscopic Subscore grading. METHODS: Here we propose an end-to-end fully automated system based on deep learning to predict a binary version of the Mayo Clinic Endoscopic Subscore directly from raw colonoscopy videos. Differently from previous studies, the proposed method mimics the assessment done in practice by a gastroenterologist, that is, traversing the whole colonoscopy video, identifying visually informative regions and computing an overall Mayo Clinic Endoscopic Subscore. The proposed deep learning-based system has been trained and deployed on raw colonoscopies using Mayo Clinic Endoscopic Subscore ground truth provided only at the colon section level, without manually selecting frames driving the severity scoring of ulcerative colitis. RESULTS AND CONCLUSION: Our evaluation on 1672 endoscopic videos obtained from a multisite data set obtained from the etrolizumab Phase II Eucalyptus and Phase III Hickory and Laurel clinical trials, show that our proposed methodology can grade endoscopic videos with a high degree of accuracy and robustness (Area Under the Receiver Operating Characteristic Curve = 0.84 for Mayo Clinic Endoscopic Subscore ⩾ 1, 0.85 for Mayo Clinic Endoscopic Subscore ⩾ 2 and 0.85 for Mayo Clinic Endoscopic Subscore ⩾ 3) and reduced amounts of manual annotation. PLAIN LANGUAGE SUMMARY: Patient, caregiver and provider thoughts on educational materials about prescribing and medication safety Artificial intelligence can be used to automatically assess full endoscopic videos and estimate the severity of ulcerative colitis. In this work, we present an artificial intelligence algorithm for the automatic grading of ulcerative colitis in full endoscopic videos. Our artificial intelligence models were trained and evaluated on a large and diverse set of colonoscopy videos obtained from concluded clinical trials. We demonstrate not only that artificial intelligence is able to accurately grade full endoscopic videos, but also that using diverse data sets obtained from multiple sites is critical to train robust AI models that could potentially be deployed on real-world data.

Discriminative and generative models for anatomical shape analysis on point clouds with deep neural networks.

We introduce deep neural networks for the analysis of anatomical shapes that learn a low-dimensional shape representation from the given task, instead of relying on hand-engineered representations. Our framework is modular and consists of several computing blocks that perform fundamental shape processing tasks. The networks operate on unordered point clouds and provide invariance to similarity transformations, avoiding the need to identify point correspondences between shapes. Based on the framework, we assemble a discriminative model for disease classification and age regression, as well as a generative model for the accruate reconstruction of shapes. In particular, we propose a conditional generative model, where the condition vector provides a mechanism to control the generative process. For instance, it enables to assess shape variations specific to a particular diagnosis, when passing it as side information. Next to working on single shapes, we introduce an extension for the joint analysis of multiple anatomical structures, where the simultaneous modeling of multiple structures can lead to a more compact encoding and a better understanding of disorders. We demonstrate the advantages of our framework in comprehensive experiments on real and synthetic data. The key insights are that (i) learning a shape representation specific to the given task yields higher performance than alternative shape descriptors, (ii) multi-structure analysis is both more efficient and more accurate than single-structure analysis, and (iii) point clouds generated by our model capture morphological differences associated to Alzheimer's disease, to the point that they can be used to train a discriminative model for disease classification. Our framework naturally scales to the analysis of large datasets, giving it the potential to learn characteristic variations in large populations.

Re-thinking the Etiological Framework of Neurodegeneration.

Neurodegenerative diseases are among the leading causes of disability and death worldwide. The disease-related socioeconomic burden is expected to increase with the steadily increasing life expectancy. In spite of decades of clinical and basic research, most strategies designed to manage degenerative brain diseases are palliative. This is not surprising as neurodegeneration progresses "silently" for decades before symptoms are noticed. Importantly, conceptual models with heuristic value used to study neurodegeneration have been constructed retrospectively, based on signs and symptoms already present in affected patients; a circumstance that may confound causes and consequences. Hence, innovative, paradigm-shifting views of the etiology of these diseases are necessary to enable their timely prevention and treatment. Here, we outline four alternative views, not mutually exclusive, on different etiological paths toward neurodegeneration. First, we propose neurodegeneration as being a secondary outcome of a primary cardiovascular cause with vascular pathology disrupting the vital homeostatic interactions between the vasculature and the brain, resulting in cognitive impairment, dementia, and cerebrovascular events such as stroke. Second, we suggest that the persistence of senescent cells in neuronal circuits may favor, together with systemic metabolic diseases, neurodegeneration to occur. Third, we argue that neurodegeneration may start in response to altered body and brain trophic interactions established via the hardwire that connects peripheral targets with central neuronal structures or by means of extracellular vesicle (EV)-mediated communication. Lastly, we elaborate on how lifespan body dysbiosis may be linked to the origin of neurodegeneration. We highlight the existence of bacterial products that modulate the gut-brain axis causing neuroinflammation and neuronal dysfunction. As a concluding section, we end by recommending research avenues to investigate these etiological paths in the future. We think that this requires an integrated, interdisciplinary conceptual research approach based on the investigation of the multimodal aspects of physiology and pathophysiology. It involves utilizing proper conceptual models, experimental animal units, and identifying currently unused opportunities derived from human data. Overall, the proposed etiological paths and experimental recommendations will be important guidelines for future cross-discipline research to overcome the translational roadblock and to develop causative treatments for neurodegenerative diseases.

Gaussian process uncertainty in age estimation as a measure of brain abnormality.

Multivariate regression models for age estimation are a powerful tool for assessing abnormal brain morphology associated to neuropathology. Age prediction models are built on cohorts of healthy subjects and are built to reflect normal aging patterns. The application of these multivariate models to diseased subjects usually results in high prediction errors, under the hypothesis that neuropathology presents a similar degenerative pattern as that of accelerated aging. In this work, we propose an alternative to the idea that pathology follows a similar trajectory than normal aging. Instead, we propose the use of metrics which measure deviations from the mean aging trajectory. We propose to measure these deviations using two different metrics: uncertainty in a Gaussian process regression model and a newly proposed age weighted uncertainty measure. Consequently, our approach assumes that pathologic brain patterns are different to those of normal aging. We present results for subjects with autism, mild cognitive impairment and Alzheimer's disease to highlight the versatility of the approach to different diseases and age ranges. We evaluate volume, thickness, and VBM features for quantifying brain morphology. Our evaluations are performed on a large number of images obtained from a variety of publicly available neuroimaging databases. Across all features, our uncertainty based measurements yield a better separation between diseased subjects and healthy individuals than the prediction error. Finally, we illustrate differences in the disease pattern to normal aging, supporting the application of uncertainty as a measure of neuropathology.

Guiding multimodal registration with learned optimization updates.

In this paper, we address the multimodal registration problem from a novel perspective, aiming to predict the transformation aligning images directly from their visual appearance. We formulate the prediction as a supervised regression task, with joint image descriptors as input and the output are the parameters of the transformation that guide the moving image towards alignment. We model the joint local appearance with context aware descriptors that capture both local and global cues simultaneously in the two modalities, while the regression function is based on the gradient boosted trees method capable of handling the very large contextual feature space. The good properties of our predictions allow us to couple them with a simple gradient-based optimization for the final registration. Our approach can be applied to any transformation parametrization as well as a broad range of modality pairs. Our method learns the relationship between the intensity distributions of a pair of modalities by using prior knowledge in the form of a small training set of aligned image pairs (in the order of 1-5 in our experiments). We demonstrate the flexibility and generality of our method by evaluating its performance on a variety of multimodal imaging pairs obtained from two publicly available datasets, RIRE (brain MR, CT and PET) and IXI (brain MR). We also show results for the very challenging deformable registration of Intravascular Ultrasound and Histology images. In these experiments, our approach has a larger capture range when compared to other state-of-the-art methods, while improving registration accuracy in complex cases.

3D intra-operative ultrasound and MR image guidance: pursuing an ultrasound-based management of brainshift to enhance neuronavigation.

BACKGROUND: Brainshift is still a major issue in neuronavigation. Incorporating intra-operative ultrasound (iUS) with advanced registration algorithms within the surgical workflow is regarded as a promising approach for a better understanding and management of brainshift. This work is intended to (1) provide three-dimensional (3D) ultrasound reconstructions specifically for brain imaging in order to detect brainshift observed intra-operatively, (2) evaluate a novel iterative intra-operative ultrasound-based deformation correction framework, and (3) validate the performance of the proposed image-registration-based deformation estimation in a clinical environment. METHODS: Eight patients with brain tumors undergoing surgical resection are enrolled in this study. For each patient, a 3D freehand iUS system is employed in combination with an intra-operative navigation (iNav) system, and intra-operative ultrasound data are acquired at three timepoints during surgery. On this foundation, we present a novel resolution-preserving 3D ultrasound reconstruction, as well as a framework to detect brainshift through iterative registration of iUS images. To validate the system, the target registration error (TRE) is evaluated for each patient, and both rigid and elastic registration algorithms are analyzed. RESULTS: The mean TRE based on 3D-iUS improves significantly using the proposed brainshift compensation compared to neuronavigation (iNav) before (2.7 vs. 5.9 mm; [Formula: see text]) and after dural opening (4.2 vs. 6.2 mm, [Formula: see text]), but not after resection (6.7 vs. 7.5 mm; [Formula: see text]). iUS depicts a significant ([Formula: see text]) dynamic spatial brainshift throughout the three timepoints. Accuracy of registration can be improved through rigid and elastic registrations by 29.2 and 33.3%, respectively, after dural opening, and by 5.2 and 0.4%, after resection. CONCLUSION: 3D-iUS systems can improve the detection of brainshift and significantly increase the accuracy of the navigation in a real scenario. 3D-iUS can thus be regarded as a robust, reliable, and feasible technology to enhance neuronavigation.

Automatic segmentation of the fetal cerebellum on ultrasound volumes, using a 3D statistical shape model.

Previous work has shown that the segmentation of anatomical structures on 3D ultrasound data sets provides an important tool for the assessment of the fetal health. In this work, we present an algorithm based on a 3D statistical shape model to segment the fetal cerebellum on 3D ultrasound volumes. This model is adjusted using an ad hoc objective function which is in turn optimized using the Nelder-Mead simplex algorithm. Our algorithm was tested on ultrasound volumes of the fetal brain taken from 20 pregnant women, between 18 and 24 gestational weeks. An intraclass correlation coefficient of 0.8528 and a mean Dice coefficient of 0.8 between cerebellar volumes measured using manual techniques and the volumes calculated using our algorithm were obtained. As far as we know, this is the first effort to automatically segment fetal intracranial structures on 3D ultrasound data.

Automatic segmentation of the cerebellum of fetuses on 3D ultrasound images, using a 3D Point Distribution Model.

Analysis of fetal biometric parameters on ultrasound images is widely performed and it is essential to estimate the gestational age, as well as the fetal growth pattern. The use of three dimensional ultrasound (3D US) is preferred over other tomographic modalities such as CT or MRI, due to its inherent safety and availability. However, the image quality of 3D US is not as good as MRI and therefore there is little work on the automatic segmentation of anatomic structures in 3D US of fetal brains. In this work we present preliminary results of the development of a 3D Point Distribution Model (PDM), for automatic segmentation, of the cerebellum in 3D US of the fetal brain. The model is adjusted to a fetal 3D ultrasound, using a genetic algorithm which optimizes a model fitting function. Preliminary results show that the approach reported is able to automatically segment the cerebellum in 3D ultrasounds of fetal brains.