Unbiased identification of novel subclinical imaging biomarkers using unsupervised deep learning.

Artificial intelligence has recently made a disruptive impact in medical imaging by successfully automatizing expert-level diagnostic tasks. However, replicating human-made decisions may inherently be biased by the fallible and dogmatic nature of human experts, in addition to requiring prohibitive amounts of training data. In this paper, we introduce an unsupervised deep learning architecture particularly designed for OCT representations for unbiased, purely data-driven biomarker discovery. We developed artificial intelligence technology that provides biomarker candidates without any restricting input or domain knowledge beyond raw images. Analyzing 54,900 retinal optical coherence tomography (OCT) volume scans of 1094 patients with age-related macular degeneration, we generated a vocabulary of 20 local and global markers capturing characteristic retinal patterns. The resulting markers were validated by linking them with clinical outcomes (visual acuity, lesion activity and retinal morphology) using correlation and machine learning regression. The newly identified features correlated well with specific biomarkers traditionally used in clinical practice (r up to 0.73), and outperformed them in correlating with visual acuity ([Formula: see text] compared to [Formula: see text] for conventional markers), despite representing an enormous compression of OCT imaging data (67 million voxels to 20 features). In addition, our method also discovered hitherto unknown, clinically relevant biomarker candidates. The presented deep learning approach identified known as well as novel medical imaging biomarkers without any prior domain knowledge. Similar approaches may be worthwhile across other medical imaging fields.

Unsupervised Identification of Disease Marker Candidates in Retinal OCT Imaging Data.

The identification and quantification of markers in medical images is critical for diagnosis, prognosis, and disease management. Supervised machine learning enables the detection and exploitation of findings that are known a priori after annotation of training examples by experts. However, supervision does not scale well, due to the amount of necessary training examples, and the limitation of the marker vocabulary to known entities. In this proof-of-concept study, we propose unsupervised identification of anomalies as candidates for markers in retinal optical coherence tomography (OCT) imaging data without a constraint to a priori definitions. We identify and categorize marker candidates occurring frequently in the data and demonstrate that these markers show a predictive value in the task of detecting disease. A careful qualitative analysis of the identified data driven markers reveals how their quantifiable occurrence aligns with our current understanding of disease course, in early- and late age-related macular degeneration (AMD) patients. A multi-scale deep denoising autoencoder is trained on healthy images, and a one-class support vector machine identifies anomalies in new data. Clustering in the anomalies identifies stable categories. Using these markers to classify healthy-, early AMD- and late AMD cases yields an accuracy of 81.40%. In a second binary classification experiment on a publicly available data set (healthy versus intermediate AMD), the model achieves an area under the ROC curve of 0.944.

Towards automatic bone age estimation from MRI: localization of 3D anatomical landmarks.

Bone age estimation (BAE) is an important procedure in forensic practice which recently has seen a shift in attention from X-ray to MRI based imaging. To automate BAE from MRI, localization of the joints between hand bones is a crucial first step, which is challenging due to anatomical variations, different poses and repeating structures within the hand. We propose a landmark localization algorithm using multiple random regression forests, first analyzing the shape of the hand from information of the whole image, thus implicitly modeling the global landmark configuration, followed by a refinement based on more local information to increase prediction accuracy. We are able to clearly outperform related approaches on our dataset of 60 T1-weighted MR images, achieving a mean landmark localization error of 1.4 ± 1.5mm, while having only 0.25% outliers with an error greater than 10mm.

Thermal effects of a combined irrigation method during implant site drilling. A standardized in vitro study using a bovine rib model.

OBJECTIVES: The purpose of this study was to evaluate the temperature changes during implant osteotomies with a combined irrigation system as compared to the commonly used external and internal irrigation under standardized conditions. MATERIAL AND METHODS: Drilling procedures were performed on VII bovine ribs using a computer-aided surgical system that ensured automated intermittent drilling cycles to simulate clinical conditions. A total of 320 drilling osteotomies were performed with twist (2 mm) and conical implant drills (3.5/4.3/5 mm) at various drilling depths (10/16 mm) and with different saline irrigation (50 ml/min) methods (without/external/internal/combined). Temperature changes were recorded in real time by two custom-built thermoprobes with 14 temperature sensors (7 sensors/thermoprobe) at defined measuring depths. RESULTS: The highest temperature increase during osteotomies was observed without any coolant irrigation (median, 8.01°C), followed by commonly used external saline irrigation (median, 2.60°C), combined irrigation (median, 1.51°C) and ultimately with internal saline irrigation (median, 1.48°C). Temperature increase with different drill diameters showed significant differences (P < 0.05) regarding drill depth, confirming drill depth and time of drilling as influencing factors of heat generation. Internal saline irrigation showed a significantly smaller temperature increase (P < 0.05) compared with combined and external irrigation. A combined irrigation procedure appears to be preferable (P < 0.05) to an external irrigation method primarily with higher osteotomy depths. CONCLUSIONS: Combined irrigation provides sufficient reduction in temperature changes during drilling, and it may be more beneficial in deeper site osteotomies. Further studies to optimize the effects of a combined irrigation are needed.

A spatio-temporal latent atlas for semi-supervised learning of fetal brain segmentations and morphological age estimation.

Prenatal neuroimaging requires reference models that reflect the normal spectrum of fetal brain development, and summarize observations from a representative sample of individuals. Collecting a sufficiently large data set of manually annotated data to construct a comprehensive in vivo atlas of rapidly developing structures is challenging but necessary for large population studies and clinical application. We propose a method for the semi-supervised learning of a spatio-temporal latent atlas of fetal brain development, and corresponding segmentations of emerging cerebral structures, such as the ventricles or cortex. The atlas is based on the annotation of a few examples, and a large number of imaging data without annotation. It models the morphological and developmental variability across the population. Furthermore, it serves as basis for the estimation of a structures' morphological age, and its deviation from the nominal gestational age during the assessment of pathologies. Experimental results covering the gestational period of 20-30 gestational weeks demonstrate segmentation accuracy achievable with minimal annotation, and precision of morphological age estimation. Age estimation results on fetuses suffering from lissencephaly demonstrate that they detect significant differences in the age offset compared to a control group.

Global localization of 3D anatomical structures by pre-filtered Hough forests and discrete optimization.

The accurate localization of anatomical landmarks is a challenging task, often solved by domain specific approaches. We propose a method for the automatic localization of landmarks in complex, repetitive anatomical structures. The key idea is to combine three steps: (1) a classifier for pre-filtering anatomical landmark positions that (2) are refined through a Hough regression model, together with (3) a parts-based model of the global landmark topology to select the final landmark positions. During training landmarks are annotated in a set of example volumes. A classifier learns local landmark appearance, and Hough regressors are trained to aggregate neighborhood information to a precise landmark coordinate position. A non-parametric geometric model encodes the spatial relationships between the landmarks and derives a topology which connects mutually predictive landmarks. During the global search we classify all voxels in the query volume, and perform regression-based agglomeration of landmark probabilities to highly accurate and specific candidate points at potential landmark locations. We encode the candidates' weights together with the conformity of the connecting edges to the learnt geometric model in a Markov Random Field (MRF). By solving the corresponding discrete optimization problem, the most probable location for each model landmark is found in the query volume. We show that this approach is able to consistently localize the model landmarks despite the complex and repetitive character of the anatomical structures on three challenging data sets (hand radiographs, hand CTs, and whole body CTs), with a median localization error of 0.80 mm, 1.19 mm and 2.71 mm, respectively.

Constructing an un-biased whole body atlas from clinical imaging data by fragment bundling.

Atlases have a tremendous impact on the study of anatomy and function, such as in neuroimaging, or cardiac analysis. They provide a means to compare corresponding measurements across populations, or model the variability in a population. Current approaches to construct atlases rely on examples that show the same anatomical structure (e.g., the brain). If we study large heterogeneous clinical populations to capture subtle characteristics of diseases, we cannot assume consistent image acquisition any more. Instead we have to build atlases from imaging data that show only parts of the overall anatomical structure. In this paper we propose a method for the automatic contruction of an un-biased whole body atlas from so-called fragments. Experimental results indicate that the fragment based atlas improves the representation accuracy of the atlas over an initial whole body template initialization.

Fully automated 3D basal ganglia activity measurement in dopamine transporter scintigraphy (Spectalyzer).

OBJECTIVES: Semiquantitative evaluation of tracer uptake in basal ganglia is superior to visual assessment of images in dopamine transporter (DAT) scintigraphy especially in follow-up of the patients. Manual drawing of regions of interest (ROIs) in two-dimensional (2D) transaxial slices of the single photon emission computed tomography (SPECT) datasets leads to a large inter- and intra-reader variability, while being time consuming. Our aim was to investigate a technique that extracts 3D ROIs in a fully automated fashion and thus might provide reproducible user-independent results allowing better follow-up control and large-scale clinical studies. METHODS: The highest activity of 123IFP-CIT is expected in the basal ganglia. The proposed method (Spectalyzer) uses the following steps to localize this maximum and extract the ROIs in 3D: (1) Dithers the SPECT volume to obtain a 3D volume with binary only. (2) Models the obtained point distributions as two multivariate Gaussian distributions and estimated their parameters using the expectation maximization algorithm. (3) Using the original SPECT activity values, thresholding is performed using a fixed percentage of maximum activity as a parameter to obtain the 3D ROIs. (4) A reference volume in the occipital region is automatically found based on the location of the two ROIs. (5) From the 3D ROIs, statistical information like mean and median activity and the volume is extracted, relative to the activity in the reference region. The resulting values are compared with values from manual 2D ROIs. Further validation is performed by means of an anthropomorphic striatal phantom. RESULTS: The method was evaluated on 12 SPECT volumes including anthropomorphic striatal phantoms. In all cases the two basal-ganglia were successfully localized and the 3D ROIs estimated, with perfect reproducibility. The obtained values for the mean activity showed the same trend with the values obtained manually and also with the results of the 2D semiautomatic software, but without the substantial inter- and intra-reader variations. CONCLUSIONS: The proposed method is successful in finding the 3D ROIs and performing the subsequent measurements automatically. It is proposed as an automatic reproducible approach for semiquantitative analysis of DAT scintigraphy.

Automatic assessment of the knee alignment angle on full-limb radiographs.

In this study a fully automatic assessment of the knee alignment angles in full-limb radiographs was developed and compared to manual standard of reference measurements in a prospective manner. The data consisted of 28 knees which were gathered from total-leg radiographs of 15 patients (12 males and 3 females with a mean age of 29.4+/-6.9 years) consecutively. For statistical evaluation, a leave-one-out cross-validation was performed. The pattern recognition and consequently the fully automatic assessment were successful in all patients. The automatically measured angles highly correlated with the standard of reference (r=0.989). The mean absolute difference was 0.578 degrees (95% CI: 0.399-0.757 degrees ). 82% of the angles differed less than 1 degrees from the standard of reference, 46% differed less than 0.5 degrees and 31% differed less than 0.2 degrees . The automatic method showed a high agreement between repeated measurements (+0.515 degrees to -0.429 degrees ). The automatic assessment of alignment angles in full-limb radiographs were equal to the manual assessment. No measurement related user interaction was necessary to achieve results.

Assessement of rheumatic diseases with computational radiology: current status and future potential.

In recent years, several computational image analysis methods to assess disease progression in rheumatic diseases were presented. This review article explains the basics of these methods as well as their potential application in rheumatic disease monitoring, it covers radiography, sonography as well as magnetic resonance imaging in quantitative analysis frameworks.

Weakly supervised group-wise model learning based on discrete optimization.

In this paper we propose a method for the weakly supervised learning of sparse appearance models from medical image data based on Markov random fields (MRF). The models are learnt from a single annotated example and additional training samples without annotations. The approach formulates the model learning as solving a set of MRFs. Both the model training and the resulting model are able to cope with complex and repetitive structures. The weakly supervised model learning yields sparse MRF appearance models that perform equally well as those trained with manual annotations, thereby eliminating the need for tedious manual training supervision. Evaluation results are reported for hand radiographs and cardiac MRI slices.

Robust autonomous model learning from 2D and 3D data sets.

In this paper we propose a weakly supervised learning algorithm for appearance models based on the minimum description length (MDL) principle. From a set of training images or volumes depicting examples of an anatomical structure, correspondences for a set of landmarks are established by group-wise registration. The approach does not require any annotation. In contrast to existing methods no assumptions about the topology of the data are made, and the topology can change throughout the data set. Instead of a continuous representation of the volumes or images, only sparse finite sets of interest points are used to represent the examples during optimization. This enables the algorithm to efficiently use distinctive points, and to handle texture variations robustly. In contrast to standard elasticity based deformation constraints the MDL criterion accounts for systematic deformations typical for training sets stemming from medical image data. Experimental results are reported for five different 2D and 3D data sets.

Object localization based on Markov random fields and symmetry interest points.

We present an approach to detect anatomical structures by configurations of interest points, from a single example image. The representation of the configuration is based on Markov Random Fields, and the detection is performed in a single iteration by the MAX-SUM algorithm. Instead of sequentially matching pairs of interest points, the method takes the entire set of points, their local descriptors and the spatial configuration into account to find an optimal mapping of modeled object to target image. The image information is captured by symmetry-based interest points and local descriptors derived from Gradient Vector Flow. Experimental results are reported for two data-sets showing the applicability to complex medical data.

Fast active appearance model search using canonical correlation analysis.

A fast AAM search algorithm based on canonical correlation analysis (CCA-AAM) is introduced. It efficiently models the dependency between texture residuals and model parameters during search. Experiments show that CCA-AAMs, while requiring similar implementation effort, consistently outperform standard search with regard to convergence speed by a factor of four.