Artificial intelligence-based fluid quantification and associated visual outcomes in a real-world, multicentre neovascular age-related macular degeneration national database.

AIM: To explore associations between artificial intelligence (AI)-based fluid compartment quantifications and 12 months visual outcomes in OCT images from a real-world, multicentre, national cohort of naïve neovascular age-related macular degeneration (nAMD) treated eyes. METHODS: Demographics, visual acuity (VA), drug and number of injections data were collected using a validated web-based tool. Fluid compartment quantifications including intraretinal fluid (IRF), subretinal fluid (SRF) and pigment epithelial detachment (PED) in the fovea (1 mm), parafovea (3 mm) and perifovea (6 mm) were measured in nanoliters (nL) using a validated AI-tool. RESULTS: 452 naïve nAMD eyes presented a mean VA gain of +5.5 letters with a median of 7 injections over 12 months. Baseline foveal IRF associated poorer baseline (44.7 vs 63.4 letters) and final VA (52.1 vs 69.1), SRF better final VA (67.1 vs 59.0) and greater VA gains (+7.1 vs +1.9), and PED poorer baseline (48.8 vs 57.3) and final VA (55.1 vs 64.1). Predicted VA gains were greater for foveal SRF (+6.2 vs +0.6), parafoveal SRF (+6.9 vs +1.3), perifoveal SRF (+6.2 vs -0.1) and parafoveal IRF (+7.4 vs +3.6, all p<0.05). Fluid dynamics analysis revealed the greatest relative volume reduction for foveal SRF (-16.4 nL, -86.8%), followed by IRF (-17.2 nL, -84.7%) and PED (-19.1 nL, -28.6%). Subgroup analysis showed greater reductions in eyes with higher number of injections. CONCLUSION: This real-world study describes an AI-based analysis of fluid dynamics and defines baseline OCT-based patient profiles that associate 12-month visual outcomes in a large cohort of treated naïve nAMD eyes nationwide.

Automated foveal location detection on spectral-domain optical coherence tomography in geographic atrophy patients.

PURPOSE: To develop a fully automated algorithm for accurate detection of fovea location in atrophic age-related macular degeneration (AMD), based on spectral-domain optical coherence tomography (SD-OCT) scans. METHODS: Image processing was conducted on a cohort of patients affected by geographic atrophy (GA). SD-OCT images (cube volume) from 55 eyes (51 patients) were extracted and processed with a layer segmentation algorithm to segment Ganglion Cell Layer (GCL) and Inner Plexiform Layer (IPL). Their en face thickness projection was convolved with a 2D Gaussian filter to find the global maximum, which corresponded to the detected fovea. The detection accuracy was evaluated by computing the distance between manual annotation and predicted location. RESULTS: The mean total location error was 0.101±0.145mm; the mean error in horizontal and vertical en face axes was 0.064±0.140mm and 0.063±0.060mm, respectively. The mean error for foveal and extrafoveal retinal pigment epithelium and outer retinal atrophy (RORA) was 0.096±0.070mm and 0.107±0.212mm, respectively. Our method obtained a significantly smaller error than the fovea localization algorithm inbuilt in the OCT device (0.313±0.283mm, p <.001) or a method based on the thinnest central retinal thickness (0.843±1.221, p <.001). Significant outliers are depicted with the reliability score of the method. CONCLUSION: Despite retinal anatomical alterations related to GA, the presented algorithm was able to detect the foveal location on SD-OCT cubes with high reliability. Such an algorithm could be useful for studying structural-functional correlations in atrophic AMD and could have further applications in different retinal pathologies.

Personalized Atrophy Risk Mapping in Age-Related Macular Degeneration.

Purpose: To develop and validate an automatic retinal pigment epithelial and outer retinal atrophy (RORA) progression prediction model for nonexudative age-related macular degeneration (AMD) cases in optical coherence tomography (OCT) scans. Methods: Longitudinal OCT data from 129 eyes/119 patients with RORA was collected and separated into training and testing groups. RORA was automatically segmented in all scans and additionally manually annotated in the test scans. OCT-based features such as layers thicknesses, mean reflectivity, and a drusen height map served as an input to the deep neural network. Based on the baseline OCT scan or the previous visit OCT, en face RORA predictions were calculated for future patient visits. The performance was quantified over time with the means of Dice scores and square root area errors. Results: The average Dice score for segmentations at baseline was 0.85. When predicting progression from baseline OCTs, the Dice scores ranged from 0.73 to 0.80 for total RORA area and from 0.46 to 0.72 for RORA growth region. The square root area error ranged from 0.13 mm to 0.33 mm. By providing continuous time output, the model enabled creation of a patient-specific atrophy risk map. Conclusions: We developed a machine learning method for RORA progression prediction, which provides continuous-time output. It was used to compute atrophy risk maps, which indicate time-to-RORA-conversion, a novel and clinically relevant way of representing disease progression. Translational Relevance: Application of recent advances in artificial intelligence to predict patient-specific progression of atrophic AMD.

Fully-automated atrophy segmentation in dry age-related macular degeneration in optical coherence tomography.

Age-related macular degeneration (AMD) is a progressive retinal disease, causing vision loss. A more detailed characterization of its atrophic form became possible thanks to the introduction of Optical Coherence Tomography (OCT). However, manual atrophy quantification in 3D retinal scans is a tedious task and prevents taking full advantage of the accurate retina depiction. In this study we developed a fully automated algorithm segmenting Retinal Pigment Epithelial and Outer Retinal Atrophy (RORA) in dry AMD on macular OCT. 62 SD-OCT scans from eyes with atrophic AMD (57 patients) were collected and split into train and test sets. The training set was used to develop a Convolutional Neural Network (CNN). The performance of the algorithm was established by cross validation and comparison to the test set with ground-truth annotated by two graders. Additionally, the effect of using retinal layer segmentation during training was investigated. The algorithm achieved mean Dice scores of 0.881 and 0.844, sensitivity of 0.850 and 0.915 and precision of 0.928 and 0.799 in comparison with Expert 1 and Expert 2, respectively. Using retinal layer segmentation improved the model performance. The proposed model identified RORA with performance matching human experts. It has a potential to rapidly identify atrophy with high consistency.

Machine Learning Can Predict Anti-VEGF Treatment Demand in a Treat-and-Extend Regimen for Patients with Neovascular AMD, DME, and RVO Associated Macular Edema.

PURPOSE: To assess the potential of machine learning to predict low and high treatment demand in real life in patients with neovascular age-related macular degeneration (nAMD), retinal vein occlusion (RVO), and diabetic macular edema (DME) treated according to a treat-and-extend regimen (TER). DESIGN: Retrospective cohort study. PARTICIPANTS: Three hundred seventy-seven eyes (340 patients) with nAMD and 333 eyes (285 patients) with RVO or DME treated with anti-vascular endothelial growth factor agents (VEGF) according to a predefined TER from 2014 through 2018. METHODS: Eyes were grouped by disease into low, moderate, and high treatment demands, defined by the average treatment interval (low, ≥10 weeks; high, ≤5 weeks; moderate, remaining eyes). Two random forest models were trained to predict the probability of the long-term treatment demand of a new patient. Both models use morphological features automatically extracted from the OCT volumes at baseline and after 2 consecutive visits, as well as patient demographic information. Evaluation of the models included a 10-fold cross-validation ensuring that no patient was present in both the training set (nAMD, approximately 339; RVO and DME, approximately 300) and test set (nAMD, approximately 38; RVO and DME, approximately 33). MAIN OUTCOME MEASURES: Mean area under the receiver operating characteristic curve (AUC) of both models; contribution to the prediction and statistical significance of the input features. RESULTS: Based on the first 3 visits, it was possible to predict low and high treatment demand in nAMD eyes and in RVO and DME eyes with similar accuracy. The distribution of low, high, and moderate demanders was 127, 42, and 208, respectively, for nAMD and 61, 50, and 222, respectively, for RVO and DME. The nAMD-trained models yielded mean AUCs of 0.79 and 0.79 over the 10-fold crossovers for low and high demand, respectively. Models for RVO and DME showed similar results, with a mean AUC of 0.76 and 0.78 for low and high demand, respectively. Even more importantly, this study revealed that it is possible to predict low demand reasonably well at the first visit, before the first injection. CONCLUSIONS: Machine learning classifiers can predict treatment demand and may assist in establishing patient-specific treatment plans in the near future.

Automated Quantification of Pathological Fluids in Neovascular Age-Related Macular Degeneration, and Its Repeatability Using Deep Learning.

Purpose: To develop a reliable algorithm for the automated identification, localization, and volume measurement of exudative manifestations in neovascular age-related macular degeneration (nAMD), including intraretinal (IRF), subretinal fluid (SRF), and pigment epithelium detachment (PED), using a deep-learning approach. Methods: One hundred seven spectral domain optical coherence tomography (OCT) cube volumes were extracted from nAMD eyes. Manual annotation of IRF, SRF, and PED was performed. Ninety-two OCT volumes served as training and validation set, and 15 OCT volumes from different patients as test set. The performance of our fluid segmentation method was quantified by means of pixel-wise metrics and volume correlations and compared to other methods. Repeatability was tested on 42 other eyes with five OCT volume scans acquired on the same day. Results: The fully automated algorithm achieved good performance for the detection of IRF, SRF, and PED. The area under the curve for detection, sensitivity, and specificity was 0.97, 0.95, and 0.99, respectively. The correlation coefficients for the fluid volumes were 0.99, 0.99, and 0.91, respectively. The Dice score was 0.73, 0.67, and 0.82, respectively. For the largest volume quartiles the Dice scores were >0.90. Including retinal layer segmentation contributed positively to the performance. The repeatability of volume prediction showed a standard deviations of 4.0 nL, 3.5 nL, and 20.0 nL for IRF, SRF, and PED, respectively. Conclusions: The deep-learning algorithm can simultaneously acquire a high level of performance for the identification and volume measurements of IRF, SRF, and PED in nAMD, providing accurate and repeatable predictions. Including layer segmentation during training and squeeze-excite block in the network architecture were shown to boost the performance. Translational Relevance: Potential applications include measurements of specific fluid compartments with high reproducibility, assistance in treatment decisions, and the diagnostic or scientific evaluation of relevant subgroups.

Comparison of Drusen Volume Assessed by Two Different OCT Devices.

To compare drusen volume between Heidelberg Spectral Domain (SD-) and Zeiss Swept-Source (SS) PlexElite Optical Coherence Tomography (OCT) determined by manual and automated segmentation methods. Thirty-two eyes of 24 patients with Age-Related Macular Degeneration (AMD) and drusen maculopathy were included. In the central 1 and 3 mm ETDRS circle drusen volumes were calculated and compared. Drusen segmentation was performed using automated manufacturer algorithms of the two OCT devices. Then, the automated segmentation was manually corrected and compared and finally analyzed using customized software. Though on SD-OCT, there was a significant difference of mean drusen volume prior to and after manual correction (mean difference: 0.0188 ± 0.0269 mm3, p < 0.001, corr. p < 0.001, correlation of r = 0.90), there was no difference found on SS-OCT (mean difference: 0.0001 ± 0.0003 mm3, p = 0.262, corr. p = 0.524, r = 1.0). Heidelberg-acquired mean drusen volume after manual correction was significantly different from Zeiss-acquired drusen volume after manual correction (mean difference: 0.1231 ± 0.0371 mm3, p < 0.001, corr. p < 0.001, r = 0.68). Using customized software, the difference of measurements between both devices decreased and correlation among the measurements improved (mean difference: 0.0547 ± 0.0744 mm3, p = 0.02, corr. p = 0.08, r = 0.937). Heidelberg SD-OCT, the Zeiss PlexElite SS-OCT, and customized software all measured significantly different drusen volumes. Therefore, devices/algorithms may not be interchangeable. Third-party customized software helps to minimize differences, which may allow a pooling of data of different devices, e.g., in multicenter trials.

Joint Segmentation and Path Classification of Curvilinear Structures.

Detection of curvilinear structures in images has long been of interest. One of the most challenging aspects of this problem is inferring the graph representation of the curvilinear network. Most existing delineation approaches first perform binary segmentation of the image and then refine it using either a set of hand-designed heuristics or a separate classifier that assigns likelihood to paths extracted from the pixel-wise prediction. In our work, we bridge the gap between segmentation and path classification by training a deep network that performs those two tasks simultaneously. We show that this approach is beneficial because it enforces consistency across the whole processing pipeline. We apply our approach on roads and neurons datasets.

Tracing in 2D to reduce the annotation effort for 3D deep delineation of linear structures.

The difficulty of obtaining annotations to build training databases still slows down the adoption of recent deep learning approaches for biomedical image analysis. In this paper, we show that we can train a Deep Net to perform 3D volumetric delineation given only 2D annotations in Maximum Intensity Projections (MIP) of the training volumes. This significantly reduces the annotation time: We conducted a user study that suggests that annotating 2D projections is on average twice as fast as annotating the original 3D volumes. Our technical contribution is a loss function that evaluates a 3D prediction against annotations of 2D projections. It is inspired by space carving, a classical approach to reconstructing complex 3D shapes from arbitrarily-positioned cameras. It can be used to train any deep network with volumetric output, without the need to change the network's architecture. Substituting the loss is all it takes to enable 2D annotations in an existing training setup. In extensive experiments on 3D light microscopy images of neurons and retinal blood vessels, and on Magnetic Resonance Angiography (MRA) brain scans, we show that, when trained on projection annotations, deep delineation networks perform as well as when they are trained using costlier 3D annotations.

Reconstructing Evolving Tree Structures in Time Lapse Sequences by Enforcing Time-Consistency.

We propose a novel approach to reconstructing curvilinear tree structures evolving over time, such as road networks in 2D aerial images or neural structures in 3D microscopy stacks acquired in vivo. To enforce temporal consistency, we simultaneously process all images in a sequence, as opposed to reconstructing structures of interest in each image independently. We formulate the problem as a Quadratic Mixed Integer Program and demonstrate the additional robustness that comes from using all available visual clues at once, instead of working frame by frame. Furthermore, when the linear structures undergo local changes over time, our approach automatically detects them.