Transfer Learning Aurora Image Classification and Magnetic Disturbance Evaluation.

We develop an open source algorithm to apply Transfer learning to Aurora image classification and Magnetic disturbance Evaluation (TAME). For this purpose, we evaluate the performance of 80 pretrained neural networks using the Oslo Auroral THEMIS (OATH) data set of all-sky images, both in terms of runtime and their features' predictive capability. From the features extracted by the best network, we retrain the last neural network layer using the Support Vector Machine (SVM) algorithm to distinguish between the labels "arc," "diffuse," "discrete," "cloud," "moon" and "clear sky/ no aurora". This transfer learning approach yields 73% accuracy in the six classes; if we aggregate the 3 auroral and 3 non-aurora classes, we achieve up to 91% accuracy. We apply our classifier to a new dataset of 550,000 images and evaluate the classifier based on these previously unseen images. To show the potential usefulness of our feature extractor and classifier, we investigate two test cases: First, we compare our predictions for the "cloudy" images to meteorological data and second we train a linear ridge model to predict perturbations in Earth's locally measured magnetic field. We demonstrate that the classifier can be used as a filter to remove cloudy images from datasets and that the extracted features allow to predict magnetometer measurements. All procedures and algorithms used in this study are publicly available, and the code and classifier are provided, which opens possibility for large scale studies of all-sky images.

Learning metal artifact reduction in cardiac CT images with moving pacemakers.

Metal objects in the human heart such as implanted pacemakers frequently lead to heavy artifacts in reconstructed CT image volumes. Due to cardiac motion, common metal artifact reduction methods which assume a static object during CT acquisition are not applicable. We propose a fully automatic Dynamic Pacemaker Artifact Reduction (DyPAR+) pipeline which is built of three convolutional neural network (CNN) ensembles. In a first step, pacemaker metal shadows are segmented directly in the raw projection data by the SegmentationNets. Second, resulting metal shadow masks are passed to the InpaintingNets which replace metal-affected line integrals in the sinogram for subsequent reconstruction of a metal-free image volume. Third, the metal locations in a pre-selected motion state are predicted by the ReinsertionNets based on a stack of partial angle back-projections generated from the segmented metal shadow mask. We generate the data required for the supervised learning processes by introducing synthetic, moving pacemaker leads into 14 clinical cases without pacemakers. The SegmentationNets and the ReinsertionNets achieve average Dice coefficients of 94.16% ± 2.01% and 55.60% ± 4.79% during testing on clinical data with synthetic metal leads. With a mean absolute reconstruction error of 11.54 HU ± 2.49 HU in the image domain, the InpaintingNets outperform the hand-crafted approaches PatchMatch and inverse distance weighting. Application of the proposed DyPAR+ pipeline to nine clinical test cases with real pacemakers leads to significant reduction of metal artifacts and demonstrates the transferability to clinical practice. Especially the SegmentationNets and InpaintingNets generalize well to unseen acquisition modes and contrast protocols.

Motion estimation and correction in cardiac CT angiography images using convolutional neural networks.

Cardiac motion artifacts frequently reduce the interpretability of coronary computed tomography angiography (CCTA) images and potentially lead to misinterpretations or preclude the diagnosis of coronary artery disease (CAD). In this paper, a novel motion compensation approach dealing with Coronary Motion estimation by Patch Analysis in CT data (CoMPACT) is presented. First, the required data for supervised learning is generated by the Coronary Motion Forward Artifact model for CT data (CoMoFACT) which introduces simulated motion to 19 artifact-free clinical CT cases with step-and-shoot acquisition protocol. Second, convolutional neural networks (CNNs) are trained to estimate underlying 2D motion vectors from 2.5D image patches based on the coronary artifact appearance. In a phantom study with computer-simulated vessels, CNNs predict the motion direction and the motion magnitude with average test accuracies of 13.37°±1.21° and 0.77 ±â€¯0.09 mm, respectively. On clinical data with simulated motion, average test accuracies of 34.85°±2.09° and 1.86 ±â€¯0.11 mm are achieved, whereby the precision of the motion direction prediction increases with the motion magnitude. The trained CNNs are integrated into an iterative motion compensation pipeline which includes distance-weighted motion vector extrapolation. Alternating motion estimation and compensation in twelve clinical cases with real cardiac motion artifacts leads to significantly reduced artifact levels, especially in image data with severe artifacts. In four observer studies, mean artifact levels of 3.08 ±â€¯0.24 without MC and 2.28 ±â€¯0.29 with CoMPACT MC are rated in a five point Likert scale.

Motion artifact recognition and quantification in coronary CT angiography using convolutional neural networks.

Excellent image quality is a primary prerequisite for diagnostic non-invasive coronary CT angiography. Artifacts due to cardiac motion may interfere with detection and diagnosis of coronary artery disease and render subsequent treatment decisions more difficult. We propose deep-learning-based measures for coronary motion artifact recognition and quantification in order to assess the diagnostic reliability and image quality of coronary CT angiography images. More specifically, the application, steering and evaluation of motion compensation algorithms can be triggered by these measures. A Coronary Motion Forward Artifact model for CT data (CoMoFACT) is developed and applied to clinical cases with excellent image quality to introduce motion artifacts using simulated motion vector fields. The data required for supervised learning is generated by the CoMoFACT from 17 prospectively ECG-triggered clinical cases with controlled motion levels on a scale of 0-10. Convolutional neural networks achieve an accuracy of 93.3% ±â€¯1.8% for the classification task of separating motion-free from motion-perturbed coronary cross-sectional image patches. The target motion level is predicted by a corresponding regression network with a mean absolute error of 1.12 ±â€¯0.07. Transferability and generalization capabilities are demonstrated by motion artifact measurements on eight additional CCTA cases with real motion artifacts.

Generating anatomical models of the heart and the aorta from medical images for personalized physiological simulations.

The anatomy and motion of the heart and the aorta are essential for patient-specific simulations of cardiac electrophysiology, wall mechanics and hemodynamics. Within the European integrated project euHeart, algorithms have been developed that allow to efficiently generate patient-specific anatomical models from medical images from multiple imaging modalities. These models, for instance, account for myocardial deformation, cardiac wall motion, and patient-specific tissue information like myocardial scar location. Furthermore, integration of algorithms for anatomy extraction and physiological simulations has been brought forward. Physiological simulations are linked closer to anatomical models by encoding tissue properties, like the muscle fibers, into segmentation meshes. Biophysical constraints are also utilized in combination with image analysis to assess tissue properties. Both examples show directions of how physiological simulations could provide new challenges and stimuli for image analysis research in the future.