SEMPAI: a Self-Enhancing Multi-Photon Artificial Intelligence for Prior-Informed Assessment of Muscle Function and Pathology.

Deep learning (DL) shows notable success in biomedical studies. However, most DL algorithms work as black boxes, exclude biomedical experts, and need extensive data. This is especially problematic for fundamental research in the laboratory, where often only small and sparse data are available and the objective is knowledge discovery rather than automation. Furthermore, basic research is usually hypothesis-driven and extensive prior knowledge (priors) exists. To address this, the Self-Enhancing Multi-Photon Artificial Intelligence (SEMPAI) that is designed for multiphoton microscopy (MPM)-based laboratory research is presented. It utilizes meta-learning to optimize prior (and hypothesis) integration, data representation, and neural network architecture simultaneously. By this, the method allows hypothesis testing with DL and provides interpretable feedback about the origin of biological information in 3D images. SEMPAI performs multi-task learning of several related tasks to enable prediction for small datasets. SEMPAI is applied on an extensive MPM database of single muscle fibers from a decade of experiments, resulting in the largest joint analysis of pathologies and function for single muscle fibers to date. It outperforms state-of-the-art biomarkers in six of seven prediction tasks, including those with scarce data. SEMPAI's DL models with integrated priors are superior to those without priors and to prior-only approaches.

How scan parameter choice affects deep learning-based coronary artery disease assessment from computed tomography.

Recently, algorithms capable of assessing the severity of Coronary Artery Disease (CAD) in form of the Coronary Artery Disease-Reporting and Data System (CAD-RADS) grade from Coronary Computed Tomography Angiography (CCTA) scans using Deep Learning (DL) were proposed. Before considering to apply these algorithms in clinical practice, their robustness regarding different commonly used Computed Tomography (CT)-specific image formation parameters-including denoising strength, slab combination, and reconstruction kernel-needs to be evaluated. For this study, we reconstructed a data set of 500 patient CCTA scans under seven image formation parameter configurations. We select one default configuration and evaluate how varying individual parameters impacts the performance and stability of a typical algorithm for automated CAD assessment from CCTA. This algorithm consists of multiple preprocessing and a DL prediction step. We evaluate the influence of the parameter changes on the entire pipeline and additionally on only the DL step by propagating the centerline extraction results of the default configuration to all others. We consider the standard deviation of the CAD severity prediction grade difference between the default and variation configurations to assess the stability w.r.t. parameter changes. For the full pipeline we observe slight instability (± 0.226 CAD-RADS) for all variations. Predictions are more stable with centerlines propagated from the default to the variation configurations (± 0.122 CAD-RADS), especially for differing denoising strengths (± 0.046 CAD-RADS). However, stacking slabs with sharp boundaries instead of mixing slabs in overlapping regions (called true stack ± 0.313 CAD-RADS) and increasing the sharpness of the reconstruction kernel (± 0.150 CAD-RADS) leads to unstable predictions. Regarding the clinically relevant tasks of excluding CAD (called rule-out; AUC default 0.957, min 0.937) and excluding obstructive CAD (called hold-out; AUC default 0.971, min 0.964) the performance remains on a high level for all variations. Concluding, an influence of reconstruction parameters on the predictions is observed. Especially, scans reconstructed with the true stack parameter need to be treated with caution when using a DL-based method. Also, reconstruction kernels which are underrepresented in the training data increase the prediction uncertainty.

Myofibrillar Lattice Remodeling Is a Structural Cytoskeletal Predictor of Diaphragm Muscle Weakness in a Fibrotic mdx (mdx Cmah-/-) Model.

Duchenne muscular dystrophy (DMD) is a degenerative genetic myopathy characterized by complete absence of dystrophin. Although the mdx mouse lacks dystrophin, its phenotype is milder compared to DMD patients. The incorporation of a null mutation in the Cmah gene led to a more DMD-like phenotype (i.e., more fibrosis). Although fibrosis is thought to be the major determinant of 'structural weakness', intracellular remodeling of myofibrillar geometry was shown to be a major cellular determinant thereof. To dissect the respective contribution to muscle weakness, we assessed biomechanics and extra- and intracellular architecture of whole muscle and single fibers from extensor digitorum longus (EDL) and diaphragm. Despite increased collagen contents in both muscles, passive stiffness in mdx Cmah-/- diaphragm was similar to wt mice (EDL muscles were twice as stiff). Isometric twitch and tetanic stresses were 50% reduced in mdx Cmah-/- diaphragm (15% in EDL). Myofibrillar architecture was severely compromised in mdx Cmah-/- single fibers of both muscle types, but more pronounced in diaphragm. Our results show that the mdx Cmah-/- genotype reproduces DMD-like fibrosis but is not associated with changes in passive visco-elastic muscle stiffness. Furthermore, detriments in active isometric force are compatible with the pronounced myofibrillar disarray of the dystrophic background.

Label-free analysis of inflammatory tissue remodeling in murine lung tissue based on multiphoton microscopy, Raman spectroscopy and machine learning.

Inflammatory fibrotic tissue remodeling can lead to severe morbidity. Histopathology grading requires extraction of biopsies and elaborate tissue processing. Label-free optical technologies can provide diagnostic readout without preparation and artificial stainings and show potential for in vivo applications. Here, we present an integration of Raman spectroscopy (RS) and multiphoton microscopy for joint investigation of the bio-chemical composition and morphological features related to cellular components and connective tissue. Both modalities show that collagen signatures were significantly increased in a murine fibrosis model. Furthermore, autofluorescence signatures assigned to immune cells show high correlation with disease severity. RS indicates increased levels of elastin and lipids. Further, we investigated the effect of joint data sets on prediction performance in machine learning models. Although binary classification did not benefit from adding more features, multi-class classification was improved by integrated data sets.

Radiomics Features of the Spleen as Surrogates for CT-Based Lymphoma Diagnosis and Subtype Differentiation.

The spleen is often involved in malignant lymphoma, which manifests on CT as either splenomegaly or focal, hypodense lymphoma lesions. This study aimed to investigate the diagnostic value of radiomics features of the spleen in classifying malignant lymphoma against non-lymphoma as well as the determination of malignant lymphoma subtypes in the case of disease presence-in particular Hodgkin lymphoma (HL), diffuse large B-cell lymphoma (DLBCL), mantle-cell lymphoma (MCL), and follicular lymphoma (FL). Spleen segmentations of 326 patients (139 female, median age 54.1 +/- 18.7 years) were generated and 1317 radiomics features per patient were extracted. For subtype classification, we created four different binary differentiation tasks and addressed them with a Random Forest classifier using 10-fold cross-validation. To detect the most relevant features, permutation importance was analyzed. Classifier results using all features were: malignant lymphoma vs. non-lymphoma AUC = 0.86 (p < 0.01); HL vs. NHL AUC = 0.75 (p < 0.01); DLBCL vs. other NHL AUC = 0.65 (p < 0.01); MCL vs. FL AUC = 0.67 (p < 0.01). Classifying malignant lymphoma vs. non-lymphoma was also possible using only shape features AUC = 0.77 (p < 0.01), with the most important feature being sphericity. Based on only shape features, a significant AUC could be achieved for all tasks, however, best results were achieved combining shape and textural features. This study demonstrates the value of splenic imaging and radiomic analysis in the diagnostic process in malignant lymphoma detection and subtype classification.

Quantitative Imaging Biomarkers of the Whole Liver Tumor Burden Improve Survival Prediction in Metastatic Pancreatic Cancer.

Finding prognostic biomarkers with high accuracy in patients with pancreatic cancer (PC) remains a challenging problem. To improve the prediction of survival and to investigate the relevance of quantitative imaging biomarkers (QIB) we combined QIB with established clinical parameters. In this retrospective study a total of 75 patients with metastatic PC and liver metastases were analyzed. Segmentations of whole liver tumor burden (WLTB) from baseline contrast-enhanced CT images were used to derive QIBs. The benefits of QIBs in multivariable Cox models were analyzed in comparison with two clinical prognostic models from the literature. To discriminate survival, the two clinical models had concordance indices of 0.61 and 0.62 in a statistical setting. Combined clinical and imaging-based models achieved concordance indices of 0.74 and 0.70 with WLTB volume, tumor burden score (TBS), and bilobar disease being the three WLTB parameters that were kept by backward elimination. These combined clinical and imaging-based models have significantly higher predictive performance in discriminating survival than the underlying clinical models alone (p < 0.003). Radiomics and geometric WLTB analysis of patients with metastatic PC with liver metastases enhances the modeling of survival compared with models based on clinical parameters alone.

Unraveling the interplay of image formation, data representation and learning in CT-based COPD phenotyping automation: The need for a meta-strategy.

PURPOSE: In the literature on automated phenotyping of chronic obstructive pulmonary disease (COPD), there is a multitude of isolated classical machine learning and deep learning techniques, mostly investigating individual phenotypes, with small study cohorts and heterogeneous meta-parameters, e.g., different scan protocols or segmented regions. The objective is to compare the impact of different experimental setups, i.e., varying meta-parameters related to image formation and data representation, with the impact of the learning technique for subtyping automation for a variety of phenotypes. The identified associations of these parameters with automation performance and their interactions might be a first step towards a determination of optimal meta-parameters, i.e., a meta-strategy. METHODS: A clinical cohort of 981 patients (53.8 ± 15.1 years, 554 male) was examined. The inspiratory CT images were analyzed to automate the diagnosis of 13 COPD phenotypes given by two radiologists. A benchmark feature set that integrates many quantitative criteria was extracted from the lung and trained a variety of learning algorithms on the first 654 patients (two thirds) and the respective algorithm retrospectively assessed the remaining 327 patients (one third). The automation performance was evaluated by the area under the receiver operating characteristic curve (AUC). 1717 experiments were conducted with varying meta-parameters such as reconstruction kernel, segmented regions and input dimensionality, i.e., number of extracted features. The association of the meta-parameters with the automation performance was analyzed by multivariable general linear model decomposition of the automation performance in the contributions of meta-parameters and the learning technique. RESULTS: The automation performance varied strongly for varying meta-parameters. For emphysema-predominant phenotypes, an AUC of 93%-95% could be achieved for the best meta-configuration. The airways-predominant phenotypes led to a lower performance of 65%-85%, while smooth kernel configurations on average were unexpectedly superior to those with sharp kernels. The performance impact of meta-parameters, even that of often neglected ones like the missing-data imputation, was in general larger than that of the learning technique. Advanced learning techniques like 3D deep learning or automated machine learning yielded inferior automation performance for non-optimal meta-configurations in comparison to simple techniques with suitable meta-configurations. The best automation performance was achieved by a combination of modern learning techniques and a suitable meta-configuration. CONCLUSIONS: Our results indicate that for COPD phenotype automation, study design parameters such as reconstruction kernel and the model input dimensionality should be adapted to the learning technique and may be more important than the technique itself. To achieve optimal automation and prediction results, the interaction between input those meta-parameters and the learning technique should be considered. This might be particularly relevant for the development of specific scan protocols for novel learning algorithms, and towards an understanding of good study design for automated phenotyping.

The relevance of CT-based geometric and radiomics analysis of whole liver tumor burden to predict survival of patients with metastatic colorectal cancer.

OBJECTIVES: To investigate the prediction of 1-year survival (1-YS) in patients with metastatic colorectal cancer with use of a systematic comparative analysis of quantitative imaging biomarkers (QIBs) based on the geometric and radiomics analysis of whole liver tumor burden (WLTB) in comparison to predictions based on the tumor burden score (TBS), WLTB volume alone, and a clinical model. METHODS: A total of 103 patients (mean age: 61.0 ± 11.2 years) with colorectal liver metastases were analyzed in this retrospective study. Automatic segmentations of WLTB from baseline contrast-enhanced CT images were used. Established biomarkers as well as a standard radiomics model building were used to derive 3 prognostic models. The benefits of a geometric metastatic spread (GMS) model, the Aerts radiomics prior model of the WLTB, and the performance of TBS and WLTB volume alone were assessed. All models were analyzed in both statistical and predictive machine learning settings in terms of AUC. RESULTS: TBS showed the best discriminative performance in a statistical setting to discriminate 1-YS (AUC = 0.70, CI: [0.56, 0.90]). For the machine learning-based prediction for unseen patients, both a model of the GMS of WLTB (0.73, CI: [0.60, 0.84]) and the Aerts radiomics prior model (0.76, CI: [0.65, 0.86]) applied on the WLTB showed a numerically higher predictive performance than TBS (0.68, CI: [0.54, 0.79]), radiomics (0.65, CI: [0.55, 0.78]), WLTB volume alone (0.53, CI: [0.40. 0.66]), or the clinical model (0.56, CI: [0.43, 0.67]). CONCLUSIONS: The imaging-based GMS model may be a first step towards a more fine-grained machine learning extension of the TBS concept for risk stratification in mCRC patients without the vulnerability to technical variance of radiomics. KEY POINTS: • CT-based geometric distribution and radiomics analysis of whole liver tumor burden in metastatic colorectal cancer patients yield prognostic information. • Differences in survival are possibly attributable to the spatial distribution of metastatic lesions and the geometric metastatic spread analysis of all liver metastases may serve as robust imaging biomarker invariant to technical variation. • Imaging-based prediction models outperform clinical models for 1-year survival prediction in metastatic colorectal cancer patients with liver metastases.

Multiparametric MRI for Prostate Cancer Characterization: Combined Use of Radiomics Model with PI-RADS and Clinical Parameters.

Radiomics is an emerging field of image analysis with potential applications in patient risk stratification. This study developed and evaluated machine learning models using quantitative radiomic features extracted from multiparametric magnetic resonance imaging (mpMRI) to detect and classify prostate cancer (PCa). In total, 191 patients that underwent prostatic mpMRI and combined targeted and systematic fusion biopsy were retrospectively included. Segmentations of the whole prostate glands and index lesions were performed manually in apparent diffusion coefficient (ADC) maps and T2-weighted MRI. Radiomic features were extracted from regions corresponding to the whole prostate gland and index lesion. The best performing combination of feature setup and classifier was selected to compare its predictive ability of the radiologist's evaluation (PI-RADS), mean ADC, prostate specific antigen density (PSAD) and digital rectal examination (DRE) using receiver operating characteristic (ROC) analysis. Models were evaluated using repeated 5-fold cross-validation and a separate independent test cohort. In the test cohort, an ensemble model combining a radiomics model, with models for PI-RADS, PSAD and DRE achieved high predictive AUCs for the differentiation of (i) malignant from benign prostatic lesions (AUC = 0.889) and of (ii) clinically significant (csPCa) from clinically insignificant PCa (cisPCa) (AUC = 0.844). Our combined model was numerically superior to PI-RADS for cancer detection (AUC = 0.779; p = 0.054) as well as for clinical significance prediction (AUC = 0.688; p = 0.209) and showed a significantly better performance compared to mADC for csPCa prediction (AUC = 0.571; p = 0.022). In our study, radiomics accurately characterizes prostatic index lesions and shows performance comparable to radiologists for PCa characterization. Quantitative image data represent a potential biomarker, which, when combined with PI-RADS, PSAD and DRE, predicts csPCa more accurately than mADC. Prognostic machine learning models could assist in csPCa detection and patient selection for MRI-guided biopsy.

Publisher Correction: The Technome - A Predictive Internal Calibration Approach for Quantitative Imaging Biomarker Research.

An amendment to this paper has been published and can be accessed via a link at the top of the paper.

The Technome - A Predictive Internal Calibration Approach for Quantitative Imaging Biomarker Research.

The goal of radiomics is to convert medical images into a minable data space by extraction of quantitative imaging features for clinically relevant analyses, e.g. survival time prediction of a patient. One problem of radiomics from computed tomography is the impact of technical variation such as reconstruction kernel variation within a study. Additionally, what is often neglected is the impact of inter-patient technical variation, resulting from patient characteristics, even when scan and reconstruction parameters are constant. In our approach, measurements within 3D regions-of-interests (ROI) are calibrated by further ROIs such as air, adipose tissue, liver, etc. that are used as control regions (CR). Our goal is to derive general rules for an automated internal calibration that enhance prediction, based on the analysed features and a set of CRs. We define qualification criteria motivated by status-quo radiomics stability analysis techniques to only collect information from the CRs which is relevant given a respective task. These criteria are used in an optimisation to automatically derive a suitable internal calibration for prediction tasks based on the CRs. Our calibration enhanced the performance for centrilobular emphysema prediction in a COPD study and prediction of patients' one-year-survival in an oncological study.

Three-dimensional Distribution of Muscle and Adipose Tissue of the Thigh at CT: Association with Acute Hip Fracture.

Purpose To evaluate determinants of hip fracture by assessing soft-tissue composition of the upper thigh at CT. Materials and Methods In this retrospective analysis of prospectively collected data, CT studies in 55 female control participants (mean age, 73.1 years ± 9.3 [standard deviation]) were compared with those in 40 female patients (mean age, 80.2 years ± 11.0) with acute hip fractures. Eighty-seven descriptors of the soft-tissue composition were determined. A multivariable best subsets analysis was used to extract parameters best associated with hip fracture. Results were adjusted for age, height, and weight. Results of soft-tissue parameters were compared with bone mineral density (BMD) and cortical bone thickness. Areas under the receiver operating characteristic curve (AUCs) adjusted for multiple comparisons were determined to discriminate fracture. Results The hip fracture group was characterized by lower BMD, lower cortical thickness, lower relative adipose tissue volume of the upper thigh, and higher extramyocellular lipid (EML) surface density. The relative volume of adipose tissue combined with EML surface density (model S1) was associated with hip fracture (AUC, 0.85; 95% confidence interval [CI]: 0.78, 0.93), as well as trochanteric trabecular BMD combined with neck cortical thickness (model B2) (AUC, 0.84; 95% CI: 0.75, 0.92). The model including all four parameters provided significantly better (P < .01) discrimination (AUC, 0.92; 95% CI: 0.86, 0.97) than model S1 or B2. Conclusion In addition to bone mineral density and geometry of the proximal femur, the amount of adipose tissue of the upper thigh and the distribution of the adipocytes in the muscles are significantly associated with acute hip fracture at CT. © RSNA, 2018 Online supplemental material is available for this article.

A reproducible semi-automatic method to quantify the muscle-lipid distribution in clinical 3D CT images of the thigh.

Many studies use threshold-based techniques to assess in vivo the muscle, bone and adipose tissue distribution of the legs using computed tomography (CT) imaging. More advanced techniques divide the legs into subcutaneous adipose tissue (SAT), anatomical muscle (muscle tissue and adipocytes within the muscle border) and intra- and perimuscular adipose tissue. In addition, a so-called muscle density directly derived from the CT-values is often measured. We introduce a new integrated approach to quantify the muscle-lipid system (MLS) using quantitative CT in patients with sarcopenia or osteoporosis. The analysis targets the thigh as many CT studies of the hip do not include entire legs The framework consists of an anatomic coordinate system, allowing delineation of reproducible volumes of interest, a robust semi-automatic 3D segmentation of the fascia and a comprehensive method to quantify of the muscle and lipid distribution within the fascia. CT density-dependent features are calibrated using subject-specific internal CT values of the SAT and external CT values of an in scan calibration phantom. Robustness of the framework with respect to operator interaction, image noise and calibration was evaluated. Specifically, the impact of inter- and intra-operator reanalysis precision and addition of Gaussian noise to simulate lower radiation exposure on muscle and AT volumes, muscle density and 3D texture features quantifying MLS within the fascia, were analyzed. Existing data of 25 subjects (age: 75.6 ± 8.7) with porous and low-contrast muscle structures were included in the analysis. Intra- and inter-operator reanalysis precision errors were below 1% and mostly comparable to 1% of cohort variation of the corresponding features. Doubling the noise changed most 3D texture features by up to 15% of the cohort variation but did not affect density and volume measurements. The application of the novel technique is easy with acceptable processing time. It can thus be employed for a comprehensive quantification of the muscle-lipid system enabling radiomics approaches to musculoskeletal disorders.