AI supported fetal echocardiography with quality assessment.

This study aimed to develop a deep learning model to assess the quality of fetal echocardiography and to perform prospective clinical validation. The model was trained on data from the 18-22-week anomaly scan conducted in seven hospitals from 2008 to 2018. Prospective validation involved 100 patients from two hospitals. A total of 5363 images from 2551 pregnancies were used for training and validation. The model's segmentation accuracy depended on image quality measured by a quality score (QS). It achieved an overall average accuracy of 0.91 (SD 0.09) across the test set, with images having above-average QS scoring 0.97 (SD 0.03). During prospective validation of 192 images, clinicians rated 44.8% (SD 9.8) of images as equal in quality, 18.69% (SD 5.7) favoring auto-captured images and 36.51% (SD 9.0) preferring manually captured ones. Images with above average QS showed better agreement on segmentations (p < 0.001) and QS (p < 0.001) with fetal medicine experts. Auto-capture saved additional planes beyond protocol requirements, resulting in more comprehensive echocardiographies. Low QS had adverse effect on both model performance and clinician's agreement with model feedback. The findings highlight the importance of developing and evaluating AI models based on 'noisy' real-life data rather than pursuing the highest accuracy possible with retrospective academic-grade data.

The path toward equal performance in medical machine learning.

To ensure equitable quality of care, differences in machine learning model performance between patient groups must be addressed. Here, we argue that two separate mechanisms can cause performance differences between groups. First, model performance may be worse than theoretically achievable in a given group. This can occur due to a combination of group underrepresentation, modeling choices, and the characteristics of the prediction task at hand. We examine scenarios in which underrepresentation leads to underperformance, scenarios in which it does not, and the differences between them. Second, the optimal achievable performance may also differ between groups due to differences in the intrinsic difficulty of the prediction task. We discuss several possible causes of such differences in task difficulty. In addition, challenges such as label biases and selection biases may confound both learning and performance evaluation. We highlight consequences for the path toward equal performance, and we emphasize that leveling up model performance may require gathering not only more data from underperforming groups but also better data. Throughout, we ground our discussion in real-world medical phenomena and case studies while also referencing relevant statistical theory.

Semantic similarity metrics for image registration.

Image registration aims to find geometric transformations that align images. Most algorithmic and deep learning-based methods solve the registration problem by minimizing a loss function, consisting of a similarity metric comparing the aligned images, and a regularization term ensuring smoothness of the transformation. Existing similarity metrics like Euclidean Distance or Normalized Cross-Correlation focus on aligning pixel intensity values or correlations, giving difficulties with low intensity contrast, noise, and ambiguous matching. We propose a semantic similarity metric for image registration, focusing on aligning image areas based on semantic correspondence instead. Our approach learns dataset-specific features that drive the optimization of a learning-based registration model. We train both an unsupervised approach extracting features with an auto-encoder, and a semi-supervised approach using supplemental segmentation data. We validate the semantic similarity metric using both deep-learning-based and algorithmic image registration methods. Compared to existing methods across four different image modalities and applications, the method achieves consistently high registration accuracy and smooth transformation fields.

Multi-centre deep learning for placenta segmentation in obstetric ultrasound with multi-observer and cross-country generalization.

The placenta is crucial to fetal well-being and it plays a significant role in the pathogenesis of hypertensive pregnancy disorders. Moreover, a timely diagnosis of placenta previa may save lives. Ultrasound is the primary imaging modality in pregnancy, but high-quality imaging depends on the access to equipment and staff, which is not possible in all settings. Convolutional neural networks may help standardize the acquisition of images for fetal diagnostics. Our aim was to develop a deep learning based model for classification and segmentation of the placenta in ultrasound images. We trained a model based on manual annotations of 7,500 ultrasound images to identify and segment the placenta. The model's performance was compared to annotations made by 25 clinicians (experts, trainees, midwives). The overall image classification accuracy was 81%. The average intersection over union score (IoU) reached 0.78. The model's accuracy was lower than experts' and trainees', but it outperformed all clinicians at delineating the placenta, IoU = 0.75 vs 0.69, 0.66, 0.59. The model was cross validated on 100 2nd trimester images from Barcelona, yielding an accuracy of 76%, IoU 0.68. In conclusion, we developed a model for automatic classification and segmentation of the placenta with consistent performance across different patient populations. It may be used for automated detection of placenta previa and enable future deep learning research in placental dysfunction.

Q-space trajectory imaging with positivity constraints (QTI+).

Q-space trajectory imaging (QTI) enables the estimation of useful scalar measures indicative of the local tissue structure. This is accomplished by employing generalized gradient waveforms for diffusion sensitization alongside a diffusion tensor distribution (DTD) model. The first two moments of the underlying DTD are made available by acquisitions at low diffusion sensitivity (b-values). Here, we show that three independent conditions have to be fulfilled by the mean and covariance tensors associated with distributions of symmetric positive semidefinite tensors. We introduce an estimation framework utilizing semi-definite programming (SDP) to guarantee that these conditions are met. Applying the framework on simulated signal profiles for diffusion tensors distributed according to non-central Wishart distributions demonstrates the improved noise resilience of QTI+ over the commonly employed estimation methods. Our findings on a human brain data set also reveal pronounced improvements, especially so for acquisition protocols featuring few number of volumes. Our method's robustness to noise is expected to not only improve the accuracy of the estimates, but also enable a meaningful interpretation of contrast in the derived scalar maps. The technique's performance on shorter acquisitions could make it feasible in routine clinical practice.

Enforcing necessary non-negativity constraints for common diffusion MRI models using sum of squares programming.

In this work we investigate the use of sum of squares constraints for various diffusion-weighted MRI models, with a goal of enforcing strict, global non-negativity of the diffusion propagator. We formulate such constraints for the mean apparent propagator model and for spherical deconvolution, guaranteeing strict non-negativity of the corresponding diffusion propagators. For the cumulant expansion similar constraints cannot exist, and we instead derive a set of auxiliary constraints that are necessary but not sufficient to guarantee non-negativity. These constraints can all be verified and enforced at reasonable computational costs using semidefinite programming. By verifying our constraints on standard reconstructions of the different models, we show that currently used weak constraints are largely ineffective at ensuring non-negativity. We further show that if strict non-negativity is not enforced then estimated model parameters may suffer from significant errors, leading to serious inaccuracies in important derived quantities such as the main fiber orientations, mean kurtosis, etc. Finally, our experiments confirm that the observed constraint violations are mostly due to measurement noise, which is difficult to mitigate and suggests that properly constrained optimization should currently be considered the norm in many cases.

Training shortest-path tractography: Automatic learning of spatial priors.

Tractography is the standard tool for automatic delineation of white matter tracts from diffusion weighted images. However, the output of tractography often requires post-processing to remove false positives and ensure a robust delineation of the studied tract, and this demands expert prior knowledge. Here we demonstrate how such prior knowledge, or indeed any prior spatial information, can be automatically incorporated into a shortest-path tractography approach to produce more robust results. We describe how such a prior can be automatically generated (learned) from a population, and we demonstrate that our framework also retains support for conventional interactive constraints such as waypoint regions. We apply our approach to the open access, high quality Human Connectome Project data, as well as a dataset acquired on a typical clinical scanner. Our results show that the use of a learned prior substantially increases the overlap of tractography output with a reference atlas on both populations, and this is confirmed by visual inspection. Furthermore, we demonstrate how a prior learned on the high quality dataset significantly increases the overlap with the reference for the more typical yet lower quality data acquired on a clinical scanner. We hope that such automatic incorporation of prior knowledge and the obviation of expert interactive tract delineation on every subject, will improve the feasibility of large clinical tractography studies.

Scalable Robust Principal Component Analysis Using Grassmann Averages.

In large datasets, manual data verification is impossible, and we must expect the number of outliers to increase with data size. While principal component analysis (PCA) can reduce data size, and scalable solutions exist, it is well-known that outliers can arbitrarily corrupt the results. Unfortunately, state-of-the-art approaches for robust PCA are not scalable. We note that in a zero-mean dataset, each observation spans a one-dimensional subspace, giving a point on the Grassmann manifold. We show that the average subspace corresponds to the leading principal component for Gaussian data. We provide a simple algorithm for computing this Grassmann Average ( GA), and show that the subspace estimate is less sensitive to outliers than PCA for general distributions. Because averages can be efficiently computed, we immediately gain scalability. We exploit robust averaging to formulate the Robust Grassmann Average (RGA) as a form of robust PCA. The resulting Trimmed Grassmann Average ( TGA) is appropriate for computer vision because it is robust to pixel outliers. The algorithm has linear computational complexity and minimal memory requirements. We demonstrate TGA for background modeling, video restoration, and shadow removal. We show scalability by performing robust PCA on the entire Star Wars IV movie; a task beyond any current method. Source code is available online.

Geodesic Atlas-Based Labeling of Anatomical Trees: Application and Evaluation on Airways Extracted From CT.

We present a fast and robust atlas-based algorithm for labeling airway trees, using geodesic distances in a geometric tree-space. Possible branch label configurations for an unlabeled airway tree are evaluated using distances to a training set of labeled airway trees. In tree-space, airway tree topology and geometry change continuously, giving a natural automatic handling of anatomical differences and noise. A hierarchical approach makes the algorithm efficient, assigning labels from the trachea and downwards. Only the airway centerline tree is used, which is relatively unaffected by pathology. The algorithm is evaluated on 80 segmented airway trees from 40 subjects at two time points, labeled by three medical experts each, testing accuracy, reproducibility and robustness in patients with chronic obstructive pulmonary disease (COPD). The accuracy of the algorithm is statistically similar to that of the experts and not significantly correlated with COPD severity. The reproducibility of the algorithm is significantly better than that of the experts, and negatively correlated with COPD severity. Evaluation of the algorithm on a longitudinal set of 8724 trees from a lung cancer screening trial shows that the algorithm can be used in large scale studies with high reproducibility, and that the negative correlation of reproducibility with COPD severity can be explained by missing branches, for instance due to segmentation problems in COPD patients. We conclude that the algorithm is robust to COPD severity given equally complete airway trees, and comparable in performance to that of experts in pulmonary medicine, emphasizing the suitability of the labeling algorithm for clinical use.

Probabilistic shortest path tractography in DTI using Gaussian Process ODE solvers.

Tractography in diffusion tensor imaging estimates connectivity in the brain through observations of local diffusivity. These observations are noisy and of low resolution and, as a consequence, connections cannot be found with high precision. We use probabilistic numerics to estimate connectivity between regions of interest and contribute a Gaussian Process tractography algorithm which allows for both quantification and visualization of its posterior uncertainty. We use the uncertainty both in visualization of individual tracts as well as in heat maps of tract locations. Finally, we provide a quantitative evaluation of different metrics and algorithms showing that the adjoint metric (8] combined with our algorithm produces paths which agree most often with experts.

Effect of inspiration on airway dimensions measured in maximal inspiration CT images of subjects without airflow limitation.

OBJECTIVES: To study the effect of inspiration on airway dimensions measured in voluntary inspiration breath-hold examinations. METHODS: 961 subjects with normal spirometry were selected from the Danish Lung Cancer Screening Trial. Subjects were examined annually for five years with low-dose CT. Automated software was utilized to segment lungs and airways, identify segmental bronchi, and match airway branches in all images of the same subject. Inspiration level was defined as segmented total lung volume (TLV) divided by predicted total lung capacity (pTLC). Mixed-effects models were used to predict relative change in lumen diameter (ALD) and wall thickness (AWT) in airways of generation 0 (trachea) to 7 and segmental bronchi (R1-R10 and L1-L10) from relative changes in inspiration level. RESULTS: Relative changes in ALD were related to relative changes in TLV/pTLC, and this distensibility increased with generation (p < 0.001). Relative changes in AWT were inversely related to relative changes in TLV/pTLC in generation 3--7 (p < 0.001). Segmental bronchi were widely dispersed in terms of ALD (5.7 ± 0.7 mm), AWT (0.86 ± 0.07 mm), and distensibility (23.5 ± 7.7%). CONCLUSIONS: Subjects who inspire more deeply prior to imaging have larger ALD and smaller AWT. This effect is more pronounced in higher-generation airways. Therefore, adjustment of inspiration level is necessary to accurately assess airway dimensions. KEY POINTS: Airway lumen diameter increases and wall thickness decreases with inspiration. The effect of inspiration is greater in higher-generation (more peripheral) airways. Airways of generation 5 and beyond are as distensible as lung parenchyma. Airway dimensions measured from CT should be adjusted for inspiration level.

Toward a theory of statistical tree-shape analysis.

To develop statistical methods for shapes with a tree-structure, we construct a shape space framework for tree-shapes and study metrics on the shape space. This shape space has singularities which correspond to topological transitions in the represented trees. We study two closely related metrics on the shape space, TED and QED. QED is a quotient euclidean distance arising naturally from the shape space formulation, while TED is the classical tree edit distance. Using Gromov's metric geometry, we gain new insight into the geometries defined by TED and QED. We show that the new metric QED has nice geometric properties that are needed for statistical analysis: Geodesics always exist and are generically locally unique. Following this, we can also show the existence and generic local uniqueness of average trees for QED. TED, while having some algorithmic advantages, does not share these advantages. Along with the theoretical framework we provide experimental proof-of-concept results on synthetic data trees as well as small airway trees from pulmonary CT scans. This way, we illustrate that our framework has promising theoretical and qualitative properties necessary to build a theory of statistical tree-shape analysis.

Tree-space statistics and approximations for large-scale analysis of anatomical trees.

Statistical analysis of anatomical trees is hard to perform due to differences in the topological structure of the trees. In this paper we define statistical properties of leaf-labeled anatomical trees with geometric edge attributes by considering the anatomical trees as points in the geometric space of leaf-labeled trees. This tree-space is a geodesic metric space where any two trees are connected by a unique shortest path, which corresponds to a tree deformation. However, tree-space is not a manifold, and the usual strategy of performing statistical analysis in a tangent space and projecting onto tree-space is not available. Using tree-space and its shortest paths, a variety of statistical properties, such as mean, principal component, hypothesis testing and linear discriminant analysis can be defined. For some of these properties it is still an open problem how to compute them; others (like the mean) can be computed, but efficient alternatives are helpful in speeding up algorithms that use means iteratively, like hypothesis testing. In this paper, we take advantage of a very large dataset (N = 8016) to obtain computable approximations, under the assumption that the data trees parametrize the relevant parts of tree-space well. Using the developed approximate statistics, we illustrate how the structure and geometry of airway trees vary across a population and show that airway trees with Chronic Obstructive Pulmonary Disease come from a different distribution in tree-space than healthy ones. Software is available from http://image.diku.dk/aasa/software.php.

Geometric tree kernels: classification of COPD from airway tree geometry.

Methodological contributions: This paper introduces a family of kernels for analyzing (anatomical) trees endowed with vector valued measurements made along the tree. While state-of-the-art graph and tree kernels use combinatorial tree/graph structure with discrete node and edge labels, the kernels presented in this paper can include geometric information such as branch shape, branch radius or other vector valued properties. In addition to being flexible in their ability to model different types of attributes, the presented kernels are computationally efficient and some of them can easily be computed for large datasets (N - 10.000) of trees with 30 - 600 branches. Combining the kernels with standard machine learning tools enables us to analyze the relation between disease and anatomical tree structure and geometry. Experimental results: The kernels are used to compare airway trees segmented from low-dose CT, endowed with branch shape descriptors and airway wall area percentage measurements made along the tree. Using kernelized hypothesis testing we show that the geometric airway trees are significantly differently distributed in patients with Chronic Obstructive Pulmonary Disease (COPD) than in healthy individuals. The geometric tree kernels also give a significant increase in the classification accuracy of COPD from geometric tree structure endowed with airway wall thickness measurements in comparison with state-of-the-art methods, giving further insight into the relationship between airway wall thickness and COPD. Software: Software for computing kernels and statistical tests is available at http://image.diku.dk/aasa/software.php.

A hierarchical scheme for geodesic anatomical labeling of airway trees.

We present a fast and robust supervised algorithm for labeling anatomical airway trees, based on geodesic distances in a geometric tree-space. Possible branch label configurations for a given tree are evaluated based on distances to a training set of labeled trees. In tree-space, the tree topology and geometry change continuously, giving a natural way to automatically handle anatomical differences and noise. The algorithm is made efficient using a hierarchical approach, in which labels are assigned from the top down. We only use features of the airway centerline tree, which are relatively unaffected by pathology. A thorough leave-one-patient-out evaluation of the algorithm is made on 40 segmented airway trees from 20 subjects labeled by 2 medical experts. We evaluate accuracy, reproducibility and robustness in patients with chronic obstructive pulmonary disease (COPD). Performance is statistically similar to the inter- and intra-expert agreement, and we found no significant correlation between COPD stage and labeling accuracy.