Quantitative comparison of automated OCT and conventional FAF-based geographic atrophy measurements in the phase 3 OAKS/DERBY trials.

With the approval of the first two substances for the treatment of geographic atrophy (GA) secondary to age-related macular degeneration (AMD), a standardized monitoring of patients treated with complement inhibitors in clinical practice is needed. Optical coherence tomography (OCT) provides high-resolution access to the retinal pigment epithelium (RPE) and neurosensory layers, such as the ellipsoid zone (EZ), which further enhances the understanding of disease progression and therapeutic effects in GA compared to conventional fundus autofluorescence (FAF). In addition, artificial intelligence-based methodology allows the identification and quantification of GA-related pathology on OCT in an objective and standardized manner. The purpose of this study was to comprehensively evaluate automated OCT monitoring for GA compared to reading center-based manual FAF measurements in the largest successful phase 3 clinical trial data of complement inhibitor therapy to date. Automated OCT analysis of RPE loss showed a high and consistent correlation to manual GA measurements on conventional FAF. EZ loss on OCT was generally larger than areas of RPE loss, supporting the hypothesis that EZ loss exceeds underlying RPE loss as a fundamental pathophysiology in GA progression. Automated OCT analysis is well suited to monitor disease progression in GA patients treated in clinical practice and clinical trials.

Disease Activity and Therapeutic Response to Pegcetacoplan for Geographic Atrophy Identified by Deep Learning-Based Analysis of OCT.

PURPOSE: To quantify morphological changes of the photoreceptors (PRs) and retinal pigment epithelium (RPE) layers under pegcetacoplan therapy in geographic atrophy (GA) using deep learning-based analysis of OCT images. DESIGN: Post hoc longitudinal image analysis. PARTICIPANTS: Patients with GA due to age-related macular degeneration from 2 prospective randomized phase III clinical trials (OAKS and DERBY). METHODS: Deep learning-based segmentation of RPE loss and PR degeneration, defined as loss of the ellipsoid zone (EZ) layer on OCT, over 24 months. MAIN OUTCOME MEASURES: Change in the mean area of RPE loss and EZ loss over time in the pooled sham arms and the pegcetacoplan monthly (PM)/pegcetacoplan every other month (PEOM) treatment arms. RESULTS: A total of 897 eyes of 897 patients were included. There was a therapeutic reduction of RPE loss growth by 22% and 20% in OAKS and 27% and 21% in DERBY for PM and PEOM compared with sham, respectively, at 24 months. The reduction on the EZ level was significantly higher with 53% and 46% in OAKS and 47% and 46% in DERBY for PM and PEOM compared with sham at 24 months. The baseline EZ-RPE difference had an impact on disease activity and therapeutic response. The therapeutic benefit for RPE loss increased with larger EZ-RPE difference quartiles from 21.9%, 23.1%, and 23.9% to 33.6% for PM versus sham (all P < 0.01) and from 13.6% (P = 0.11), 23.8%, and 23.8% to 20.0% for PEOM versus sham (P < 0.01) in quartiles 1, 2, 3, and 4, respectively, at 24 months. The therapeutic reduction of EZ loss increased from 14.8% (P = 0.09), 33.3%, and 46.6% to 77.8% (P < 0.0001) between PM and sham and from 15.9% (P = 0.08), 33.8%, and 52.0% to 64.9% (P < 0.0001) between PEOM and sham for quartiles 1 to 4 at 24 months. CONCLUSIONS: Deep learning-based OCT analysis objectively identifies and quantifies PR and RPE degeneration in GA. Reductions in further EZ loss on OCT are even higher than the effect on RPE loss in phase 3 trials of pegcetacoplan treatment. The EZ-RPE difference has a strong impact on disease progression and therapeutic response. Identification of patients with higher EZ-RPE loss difference may become an important criterion for the management of GA secondary to AMD. FINANCIAL DISCLOSURE(S): Proprietary or commercial disclosure may be found after the references.

Linking disease activity with optical coherence tomography angiography in neovascular age related macular degeneration using artificial intelligence.

To investigate quantitative associations between AI-assessed disease activity and optical coherence tomography angiography (OCTA)-derived parameters in patients with neovascular age-related macular degeneration (nAMD) undergoing anti-VEGF therapy. OCTA and SD-OCT images obtained from multicenter, randomized study data were evaluated. A deep learning algorithm (RetInSight) was used to detect and quantify macular fluid on SD-OCT. Mixed effects models were applied to evaluate correlations between fluid volumes, macular neovascularization (MNV)-type and OCTA-derived MNV parameters; lesion size (LS) and vessel area (NVA). 230 patients were included. A significant positive correlation was observed between SRF and NVA (estimate = 199.8 nl/mm2, p = 0.023), while a non-significant but negative correlation was found between SRF and LS (estimate = - 71.3 nl/mm2, p = 0.126). The presence of Type I and Type II MNV was associated with significantly less intraretinal fluid (IRF) compared to Type III MNV (estimate type I:- 52.1 nl, p = 0.019; estimate type II:- 51.7 nl, p = 0.021). A significant correlation was observed between pigment epithelial detachment (PED) and the interaction between NVA and LS (estimate:28.97 nl/mm2; p = 0.012). Residual IRF at week 12 significantly correlated to baseline NVA (estimate:38.1 nl/mm2; p = 0.015) and LS (estimate:- 22.6 nl/mm2; p = 0.012). Fluid in different compartments demonstrated disparate associations with MNV OCTA features. While IRF at baseline was most pronounced in type III MNV, residual IRF was driven by neovascular MNV characteristics. Greater NVA in proportion to LS was associated with higher amounts of SRF and PED. The correlation between these parameters may represent MNV maturation and can be used as a biomarker for resolution of disease activity. AI-based OCT analysis allows for a deeper understanding of neovascular disease in AMD and the potential to adjust therapeutic strategies to optimize outcomes through precision medicine.

VALIDATION OF AN AUTOMATED FLUID ALGORITHM ON REAL-WORLD DATA OF NEOVASCULAR AGE-RELATED MACULAR DEGENERATION OVER FIVE YEARS.

BACKGROUND/PURPOSE: To apply an automated deep learning automated fluid algorithm on data from real-world management of patients with neovascular age-related macular degeneration for quantification of intraretinal/subretinal fluid volumes in optical coherence tomography images. METHODS: Data from the Vienna Imaging Biomarker Eye Study (VIBES, 2007-2018) were analyzed. Databases were filtered for treatment-naive neovascular age-related macular degeneration with a baseline optical coherence tomography and at least one follow-up and 1,127 eyes included. Visual acuity and optical coherence tomography at baseline, Months 1 to 3/Years 1 to 5, age, sex, and treatment number were included. Artificial intelligence and certified manual grading were compared in a subanalysis of 20%. Main outcome measures were fluid volumes. RESULTS: Intraretinal/subretinal fluid volumes were maximum at baseline (intraretinal fluid: 21.5/76.6/107.1 nL; subretinal fluid 13.7/86/262.5 nL in the 1/3/6-mm area). Intraretinal fluid decreased to 5 nL at M1-M3 (1-mm) and increased to 11 nL (Y1) and 16 nL (Y5). Subretinal fluid decreased to a mean of 4 nL at M1-M3 (1-mm) and remained stable below 7 nL until Y5. Intraretinal fluid was the only variable that reflected VA change over time. Comparison with human expert readings confirmed an area under the curve of >0.9. CONCLUSION: The Vienna Fluid Monitor can precisely quantify fluid volumes in optical coherence tomography images from clinical routine over 5 years. Automated tools will introduce precision medicine based on fluid guidance into real-world management of exudative disease, improving clinical outcomes while saving resources.

Automated quantification of macular fluid in retinal diseases and their response to anti-VEGF therapy.

AIM: To objectively assess disease activity and treatment response in patients with retinal vein occlusion (RVO), neovascular age-related macular degeneration (nAMD) and centre-involved diabetic macular oedema (DME), using artificial intelligence-based fluid quantification. METHODS: Posthoc analysis of 2311 patients (11 151 spectral-domain optical coherence tomography volumes) from five clinical, multicentre trials, who received a flexible antivascular endothelial growth factor (anti-VEGF) therapy over a 12-month period. Fluid volumes were measured with a deep learning algorithm at baseline/months 1, 2, 3 and 12, for three concentric circles with diameters of 1, 3 and 6 mm (fovea, paracentral ring and pericentral ring), as well as four sectors surrounding the fovea (superior, nasal, inferior and temporal). RESULTS: In each disease, at every timepoint, most intraretinal fluid (IRF) per square millimetre was present at the fovea, followed by the paracentral ring and pericentral ring (p<0.0001). While this was also the case for subretinal fluid (SRF) in RVO/DME (p<0.0001), patients with nAMD showed more SRF in the paracentral ring than at the fovea up to month 3 (p<0.0001). Between sectors, patients with RVO/DME showed the highest IRF volumes temporally (p<0.001/p<0.0001). In each disease, more SRF was consistently found inferiorly than superiorly (p<0.02). At month 1/12, we measured the following median reductions of initial fluid volumes. For IRF: RVO, 95.9%/97.7%; nAMD, 91.3%/92.8%; DME, 37.3%/69.9%. For SRF: RVO, 94.7%/97.5%; nAMD, 98.4%/99.8%; DME, 86.3%/97.5%. CONCLUSION: Fully automated localisation and quantification of IRF/SRF over time shed light on the fluid dynamics in each disease. There is a specific anatomical response of IRF/SRF to anti-VEGF therapy in all diseases studied.

The impact of structural optical coherence tomography changes on visual function in retinal vein occlusion.

PURPOSE: We aimed to determine the correlation between optical coherence tomography (OCT)- and demographic features and baseline best corrected visual acuity (BCVA) in treatment-naïve patients with retinal vein occlusion (RVO). METHODS: This was a cross-sectional posthoc analysis of OCT images that included RVO patients from two prospective, open-label, multicentre studies. The morphological grading was done manually, in the standardized setting of a reading centre. Main outcome measure was the estimated difference in Early Treatment Diabetic Retinopathy Study letters associated with each individual biomarker. RESULTS: Included were 381/301 treatment-naïve patients with BRVO/CRVO. For BRVO, statistically significant correlations with BCVA were seen for a 100 µm increase in central subfield thickness (CST; -3.1 letters), intraretinal cysts at centre point (CP; +4.1), subretinal fluid (SRF) at CP (+3.0) and hyperreflective foci (HRF) at the central B-scan (-2.2). In CRVO, a 100 µm increase in CST was associated with a loss of -3.4 letters. In the total cohort, 100 µm increase in CST, SRF at CP and HRF at the central B-scan correlated with a difference of -3.2,+3.2 and -2.0 letters. A 10-year increase in age and female gender yielded a -2.0 and -2.5 letter decrease in the total cohort. Adjusted multiple R2 for the respective group was 18.3%/26.3%/23.5%. CONCLUSIONS: Of all parameters studied, only CST and age were consistently associated with worse BCVA in treatment-naïve RVO patients. Morphology on OCT explained only a modest part of functional loss in this patient cohort.

Unbiased identification of novel subclinical imaging biomarkers using unsupervised deep learning.

Artificial intelligence has recently made a disruptive impact in medical imaging by successfully automatizing expert-level diagnostic tasks. However, replicating human-made decisions may inherently be biased by the fallible and dogmatic nature of human experts, in addition to requiring prohibitive amounts of training data. In this paper, we introduce an unsupervised deep learning architecture particularly designed for OCT representations for unbiased, purely data-driven biomarker discovery. We developed artificial intelligence technology that provides biomarker candidates without any restricting input or domain knowledge beyond raw images. Analyzing 54,900 retinal optical coherence tomography (OCT) volume scans of 1094 patients with age-related macular degeneration, we generated a vocabulary of 20 local and global markers capturing characteristic retinal patterns. The resulting markers were validated by linking them with clinical outcomes (visual acuity, lesion activity and retinal morphology) using correlation and machine learning regression. The newly identified features correlated well with specific biomarkers traditionally used in clinical practice (r up to 0.73), and outperformed them in correlating with visual acuity ([Formula: see text] compared to [Formula: see text] for conventional markers), despite representing an enormous compression of OCT imaging data (67 million voxels to 20 features). In addition, our method also discovered hitherto unknown, clinically relevant biomarker candidates. The presented deep learning approach identified known as well as novel medical imaging biomarkers without any prior domain knowledge. Similar approaches may be worthwhile across other medical imaging fields.

Characterization of Drusen and Hyperreflective Foci as Biomarkers for Disease Progression in Age-Related Macular Degeneration Using Artificial Intelligence in Optical Coherence Tomography.

Importance: The morphologic changes and their pathognomonic distribution in progressing age-related macular degeneration (AMD) are not well understood. Objectives: To characterize the pathognomonic distribution and time course of morphologic patterns in AMD and to quantify changes distinctive for progression to macular neovascularization (MNV) and macular atrophy (MA). Design, Setting, and Participants: This cohort study included optical coherence tomography (OCT) volumes from study participants with early or intermediate AMD in the fellow eye in the HARBOR (A Study of Ranibizumab Administered Monthly or on an As-needed Basis in Patients With Subfoveal Neovascular Age-Related Macular Degeneration) trial. Patients underwent imaging monthly for 2 years (July 1, 2009, to August 31, 2012) following a standardized protocol. Data analysis was performed from June 1, 2018, to January 21, 2020. Main Outcomes and Measures: To obtain topographic correspondence between patients and over time, all scans were mapped into a joint reference frame. The time of progression to MNV and MA was established, and drusen volumes and hyperreflective foci (HRF) volumes were automatically segmented in 3 dimensions using validated artificial intelligence algorithms. Topographically resolved population means of these markers were constructed by averaging quantified drusen and HRF maps in the patient subgroups. Results: Of 1097 patients enrolled in HARBOR, 518 (mean [SD] age, 78.1 [8.2] years; 309 [59.7%] female) had early or intermediate AMD in the fellow eye at baseline. During the 24-month follow-up period, 135 (26%) eyes developed MNV, 50 eyes (10%) developed MA, and 333 (64%) eyes did not progress to advanced AMD. Drusen and HRF had distinct topographic patterns. Mean drusen thickness at the fovea was 29.6 µm (95% CI, 20.2-39.0 µm) for eyes progressing to MNV, 17.2 µm (95% CI, 9.8-24.6 µm) for eyes progressing to MA, and 17.1 µm (95% CI, 12.5-21.7 µm) for eyes without disease progression. At 0.5-mm eccentricity, mean drusen thickness was 25.8 µm (95% CI, 19.1-32.5 µm) for eyes progressing to MNV, 21.7 µm (95% CI, 14.6-28.8 µm) for eyes progressing to MA, and 14.4 µm (95% CI, 11.2-17.6 µm) for eyes without disease progression. The mean HRF thickness at the foveal center was 0.072 µm (95% CI, 0-0.152 µm) for eyes progressing to MNV, 0.059 µm (95% CI, 0-0.126 µm) for eyes progressing to MA, and 0.044 µm (95% CI, 0.007-0.081) for eyes without disease progression. At 0.5-mm eccentricity, the largest mean HRF thickness was seen in eyes progressing to MA (0.227 µm; 95% CI, 0.104-0.349 µm) followed by eyes progressing to MNV (0.161 µm; 95% CI, 0.101-0.221 µm) and eyes without disease progression (0.085 µm; 95% CI, 0.058-0.112 µm). Conclusions and Relevance: In this study, drusen and HRF represented imaging biomarkers of disease progression in AMD, demonstrating distinct topographic patterns over time that differed between eyes progressing to MNV, eyes progressing to MA, or eyes without disease progression. Automated localization and precise quantification of these factors may help to develop reliable methods of predicting future disease progression.

Effect of posterior vitreous detachment on treat-and-extend versus monthly ranibizumab for neovascular age-related macular degeneration.

AIMS: To investigate the impact of posterior vitreous detachment (PVD) on the efficacy of treat-and-extend (T&E) ranibizumab in neovascular age-related macular degeneration. METHODS: In a post hoc analysis of a randomised controlled clinical trial, spectral-domain optical coherence tomography images of treatment-naïve patients randomised to receive T&E (n=265) or monthly (n=264) ranibizumab for 12 months were included. Certified, masked graders diagnosed the presence or the absence of complete PVD. The main outcome measures were the mean change in best-corrected visual acuity (BCVA) and central retinal thickness (CRT) at month 12, the number of administered ranibizumab injections and the proportion of patients extended to more than 8 weeks. RESULTS: At baseline, complete PVD was present in 51% and 56% of patients in the monthly and T&E arms, respectively. Mean change in BCVA at month 12 was +9.0 (PVD) vs +9.5 letters (no PVD, p=0.78) in monthly treated eyes, and +6.0 (PVD) vs +7.5 letters (no PVD, p=0.42) in T&E treated eyes. Conversely, mean change in CRT at month 12 was -174 (PVD) vs -173 µm (no PVD, p=0.98) in the monthly arm, and -175 (PVD) vs -164 µm (no PVD, p=0.58) in the T&E arm. In T&E treated patients, the median number of injections was eight vs nine (p=0.035). 71% of PVD eyes were extended successfully, compared with 55% of eyes without PVD (p=0.005). CONCLUSION: PVD was not found to impact functional and anatomical outcomes of T&E ranibizumab therapy. However, patients without a complete PVD required more retreatments and were significantly less likely to be successfully extended. TRIAL REGISTRATION NUMBER: NCT01948830.

TOPOGRAPHIC ANALYSIS OF PHOTORECEPTOR LOSS CORRELATED WITH DISEASE MORPHOLOGY IN NEOVASCULAR AGE-RELATED MACULAR DEGENERATION.

PURPOSE: To quantify morphologic photoreceptor integrity during anti-vascular endothelial growth factor (anti-VEGF) therapy of neovascular age-related macular degeneration and correlate these findings with disease morphology and function. METHODS: This presents a post hoc analysis on spectral-domain optical coherence tomography data of 185 patients, acquired at baseline, Month 3, and Month 12 in a multicenter, prospective trial. Loss of the ellipsoid zone (EZ) was manually quantified in all optical coherence tomography volumes. Intraretinal cystoid fluid, subretinal fluid (SRF), and pigment epithelial detachments were automatically segmented in the full volumes using validated deep learning methods. Spatiotemporal correlation of fluid markers with EZ integrity as well as bivariate analysis between EZ integrity and best-corrected visual acuity was performed. RESULTS: At baseline, EZ integrity was predominantly impaired in the fovea, showing progressive recovery during anti-vascular endothelial growth factor therapy. Topographic analysis at baseline revealed EZ integrity to be more likely intact in areas with SRF and vice versa. Moreover, we observed a correlation between EZ integrity and resolution of SRF. Foveal EZ integrity correlated with best-corrected visual acuity at all timepoints. CONCLUSION: Improvement of EZ integrity during anti-VEGF therapy of neovascular age-related macular degeneration occurred predominantly in the fovea. Photoreceptor integrity correlated with best-corrected visual acuity. Ellipsoid zone integrity was preserved in areas of SRF and showed deterioration upon SRF resolution.

MORPHOLOGICAL AND FUNCTIONAL CHARACTERISTICS AT THE ONSET OF EXUDATIVE CONVERSION IN AGE-RELATED MACULAR DEGENERATION.

PURPOSE: To characterize retinal morphology differences among different types of choroidal neovascularization and visual function changes at the onset of exudative age-related macular degeneration. METHODS: In a post hoc analysis of a prospective clinical study, 1,097 fellow eyes from subjects with choroidal neovascularization in the study eye enrolled in the HARBOR trial were evaluated. The onset of exudation was diagnosed on monthly optical coherence tomography by two masked graders. At conversion as well as 1 month earlier, pigment epithelial detachment, intraretinal cystoid fluid, subretinal fluid, subretinal hyperreflective material, as well as ellipsoid zone and external limiting membrane loss were quantitatively analyzed. Hyperreflective foci, retinal pigment epithelial defects, haze and vitreoretinal interface status were evaluated qualitatively. Main outcome measures included visual acuity and rates of morphologic features at conversion and 1 month earlier. RESULTS: New-onset exudation was detected in 92 eyes. One month before conversion, hyperreflective foci, pigment epithelial detachment, retinal pigment epithelial defects, and haze were present in the majority of eyes. At the onset of exudation, the volumes of intraretinal cystoid fluid, subretinal fluid, subretinal hyperreflective material and pigment epithelial detachment, and the areas of external limiting membrane and ellipsoid zone loss significantly increased. The mean vision loss was -2.2 letters. Pathognomonic patterns of the different choroidal neovascularization types were already apparent 1 month before conversion. CONCLUSION: Characteristic choroidal neovascularization-associated morphological changes are preceding disease conversion, while vision loss at the onset of exudation is minimal. Individual lesion types are related to specific changes in optical coherence tomography morphology already before the time of conversion. Our findings may help to elucidate the pathophysiology of neovascular age-related macular degeneration and support the diagnosis of imminent disease conversion.

Artificial intelligence in retina.

Major advances in diagnostic technologies are offering unprecedented insight into the condition of the retina and beyond ocular disease. Digital images providing millions of morphological datasets can fast and non-invasively be analyzed in a comprehensive manner using artificial intelligence (AI). Methods based on machine learning (ML) and particularly deep learning (DL) are able to identify, localize and quantify pathological features in almost every macular and retinal disease. Convolutional neural networks thereby mimic the path of the human brain for object recognition through learning of pathological features from training sets, supervised ML, or even extrapolation from patterns recognized independently, unsupervised ML. The methods of AI-based retinal analyses are diverse and differ widely in their applicability, interpretability and reliability in different datasets and diseases. Fully automated AI-based systems have recently been approved for screening of diabetic retinopathy (DR). The overall potential of ML/DL includes screening, diagnostic grading as well as guidance of therapy with automated detection of disease activity, recurrences, quantification of therapeutic effects and identification of relevant targets for novel therapeutic approaches. Prediction and prognostic conclusions further expand the potential benefit of AI in retina which will enable personalized health care as well as large scale management and will empower the ophthalmologist to provide high quality diagnosis/therapy and successfully deal with the complexity of 21st century ophthalmology.

Prediction of Individual Disease Conversion in Early AMD Using Artificial Intelligence.

Purpose: While millions of individuals show early age-related macular degeneration (AMD) signs, yet have excellent vision, the risk of progression to advanced AMD with legal blindness is highly variable. We suggest means of artificial intelligence to individually predict AMD progression. Methods: In eyes with intermediate AMD, progression to the neovascular type with choroidal neovascularization (CNV) or the dry type with geographic atrophy (GA) was diagnosed based on standardized monthly optical coherence tomography (OCT) images by independent graders. We obtained automated volumetric segmentation of outer neurosensory layers and retinal pigment epithelium, drusen, and hyperreflective foci by spectral domain-OCT image analysis. Using imaging, demographic, and genetic input features, we developed and validated a machine learning-based predictive model assessing the risk of conversion to advanced AMD. Results: Of a total of 495 eyes, 159 eyes (32%) had converted to advanced AMD within 2 years, 114 eyes progressed to CNV, and 45 to GA. Our predictive model differentiated converting versus nonconverting eyes with a performance of 0.68 and 0.80 for CNV and GA, respectively. The most critical quantitative features for progression were outer retinal thickness, hyperreflective foci, and drusen area. The features for conversion showed pathognomonic patterns that were distinctly different for the neovascular and the atrophic pathways. Predictive hallmarks for CNV were mostly drusen-centric, while GA markers were associated with neurosensory retina and age. Conclusions: Artificial intelligence with automated analysis of imaging biomarkers allows personalized prediction of AMD progression. Moreover, pathways of progression may be specific in respect to the neovascular/atrophic type.

Fully Automated Detection and Quantification of Macular Fluid in OCT Using Deep Learning.

PURPOSE: Development and validation of a fully automated method to detect and quantify macular fluid in conventional OCT images. DESIGN: Development of a diagnostic modality. PARTICIPANTS: The clinical dataset for fluid detection consisted of 1200 OCT volumes of patients with neovascular age-related macular degeneration (AMD, n = 400), diabetic macular edema (DME, n = 400), or retinal vein occlusion (RVO, n = 400) acquired with Zeiss Cirrus (Carl Zeiss Meditec, Dublin, CA) (n = 600) or Heidelberg Spectralis (Heidelberg Engineering, Heidelberg, Germany) (n = 600) OCT devices. METHODS: A method based on deep learning to automatically detect and quantify intraretinal cystoid fluid (IRC) and subretinal fluid (SRF) was developed. The performance of the algorithm in accurately identifying fluid localization and extent was evaluated against a manual consensus reading of 2 masked reading center graders. MAIN OUTCOME MEASURES: Performance of a fully automated method to accurately detect, differentiate, and quantify intraretinal and SRF using area under the receiver operating characteristics curves, precision, and recall. RESULTS: The newly designed, fully automated diagnostic method based on deep learning achieved optimal accuracy for the detection and quantification of IRC for all 3 macular pathologies with a mean accuracy (AUC) of 0.94 (range, 0.91-0.97), a mean precision of 0.91, and a mean recall of 0.84. The detection and measurement of SRF were also highly accurate with an AUC of 0.92 (range, 0.86-0.98), a mean precision of 0.61, and a mean recall of 0.81, with superior performance in neovascular AMD and RVO compared with DME, which was represented rarely in the population studied. High linear correlation was confirmed between automated and manual fluid localization and quantification, yielding an average Pearson's correlation coefficient of 0.90 for IRC and of 0.96 for SRF. CONCLUSIONS: Deep learning in retinal image analysis achieves excellent accuracy for the differential detection of retinal fluid types across the most prevalent exudative macular diseases and OCT devices. Furthermore, quantification of fluid achieves a high level of concordance with manual expert assessment. Fully automated analysis of retinal OCT images from clinical routine provides a promising horizon in improving accuracy and reliability of retinal diagnosis for research and clinical practice in ophthalmology.

Machine Learning to Analyze the Prognostic Value of Current Imaging Biomarkers in Neovascular Age-Related Macular Degeneration.

PURPOSE: To evaluate the potential of machine learning to predict best-corrected visual acuity (BCVA) outcomes from structural and functional assessments during the initiation phase in patients receiving standardized ranibizumab therapy for neovascular age-related macular degeneration (AMD). DESIGN: Post hoc analysis of a randomized, prospective clinical trial. PARTICIPANTS: Data of 614 evaluable patients receiving intravitreal ranibizumab monthly or pro re nata according to protocol-specified criteria in the HARBOR trial. METHODS: Monthly spectral-domain (SD) OCT volume scans were processed by validated, fully automated computational image analysis. This system performs spatially resolved 3-dimensional segmentation of retinal layers, intraretinal cystoid fluid (IRF), subretinal fluid (SRF), and pigment epithelial detachments (PED). The extracted quantitative OCT biomarkers and BCVA measurements at baseline and months 1, 2, and 3 were used to predict BCVA at 12 months using random forest machine learning. This approach was also used to correlate OCT morphology to BCVA at baseline (structure-function correlation). MAIN OUTCOME MEASURES: Accuracy (R2) of the prediction models; ranking of input variables. RESULTS: Computational image analysis enabled fully automated quantitative characterization of neovascular lesions in a large-scale clinical SD-OCT data set. At baseline, OCT features and BCVA were correlated with R2 = 0.21. The most relevant biomarker for BCVA was the horizontal extension of IRF in the foveal region, whereas SRF and PED ranked low. In predicting functional outcomes, the model's accuracy increased in a linear fashion with each month. If only the baseline visit was considered, the accuracy was R2 = 0.34. At month 3, final visual acuity outcomes could be predicted with an accuracy of R2 = 0.70. The strongest predictive factor for functional outcomes at 1 year was consistently the individual BCVA level during the initiation phase. CONCLUSIONS: In this large-scale study based on a wide spectrum of morphologic and functional features, baseline BCVA correlated modestly with baseline SD-OCT, whereas functional outcomes were determined by BCVA levels during the initiation phase with a minor influence of fluid-related features. This finding suggests a re-evaluation of current diagnostic imaging features and a search for novel imaging approaches, where machine learning is a promising approach.

Spatial Correspondence Between Intraretinal Fluid, Subretinal Fluid, and Pigment Epithelial Detachment in Neovascular Age-Related Macular Degeneration.

Purpose: To identify the spatial distribution of exudative features of choroidal neovascularization in neovascular age-related macular degeneration (nAMD) based on the localization of intraretinal cystoid fluid (IRC), subretinal fluid (SRF), and pigment-epithelial detachment (PED). Methods: This retrospective cross-sectional study included spectral-domain optical coherence tomography volume scans (6 × 6 mm) of 1341 patients with treatment-naïve nAMD. IRC, SRF, and PED were detected on a per-voxel basis using fully automated segmentation algorithms. Two subsets of 37 volumes each were manually segmented to validate the automated results. The spatial correspondence of components was quantified by computing proportions of IRC-, SRF-, or PED-presenting A-scans simultaneously affected by the respective other pathomorphologic components on a per-patient basis. The median across the population is reported. Odds ratios between pairs of lesions were calculated and tested for significance pixel wise. Results: Automated image segmentation was successful in 1182 optical coherence tomography volumes, yielding more than 61 million A-scans for analysis. Overall, 81% of eyes showed IRC, 95% showed SRF, and 92% showed PED. IRC-presenting A-scans also showed SRF in a median 2.5%, PED in 32.9%. Of the SRF-presenting A-scans, 0.3% demonstrated IRC, 1.4% PED. Of the PED-presenting A-scans, 5.2% contained IRC, 2.0% SRF. Similar patterns were observed in the manually segmented subsets and via pixel-wise odds ratio analysis. Conclusions: Automated analyses of large-scale datasets in a cross-sectional study of 1182 patients with active treatment-naïve nAMD demonstrated low spatial correlation of SRF with IRC and PED in contrast to increased colocalization of IRC and PED. These morphological associations may contribute to our understanding of functional deficits in nAMD.

Prediction of Anti-VEGF Treatment Requirements in Neovascular AMD Using a Machine Learning Approach.

Purpose: The purpose of this study was to predict low and high anti-VEGF injection requirements during a pro re nata (PRN) treatment, based on sets of optical coherence tomography (OCT) images acquired during the initiation phase in neovascular AMD. Methods: Two-year clinical trial data of subjects receiving PRN ranibizumab according to protocol specified criteria in the HARBOR study after three initial monthly injections were included. OCT images were analyzed at baseline, month 1, and month 2. Quantitative spatio-temporal features computed from automated segmentation of retinal layers and fluid-filled regions were used to describe the macular microstructure. In addition, best-corrected visual acuity and demographic characteristics were included. Patients were grouped into low and high treatment categories based on first and third quartile, respectively. Random forest classification was used to learn and predict treatment categories and was evaluated with cross-validation. Results: Of 317 evaluable subjects, 71 patients presented low (≤5), 176 medium, and 70 high (≥16) injection requirements during the PRN maintenance phase from month 3 to month 23. Classification of low and high treatment requirement subgroups demonstrated an area under the receiver operating characteristic curve of 0.7 and 0.77, respectively. The most relevant feature for prediction was subretinal fluid volume in the central 3 mm, with the highest predictive values at month 2. Conclusions: We proposed and evaluated a machine learning methodology to predict anti-VEGF treatment needs from OCT scans taken during treatment initiation. The results of this pilot study are an important step toward image-guided prediction of treatment intervals in the management of neovascular AMD.

Computational image analysis for prognosis determination in DME.

In this pilot study, we evaluated the potential of computational image analysis of optical coherence tomography (OCT) data to determine the prognosis of patients with diabetic macular edema (DME). Spectral-domain OCT scans with fully automated retinal layer segmentation and segmentation of intraretinal cystoid fluid (IRC) and subretinal fluid of 629 patients receiving anti-vascular endothelial growth factor therapy for DME in a randomized prospective clinical trial were analyzed. The results were used to define 312 potentially predictive features at three timepoints (baseline, weeks 12 and 24) for best-corrected visual acuity (BCVA) at baseline and after one year used in a random forest prediction path. Preliminarily, IRC in the outer nuclear layer in the 3-mm area around the fovea seemed to have the greatest predictive value for BCVA at baseline, and IRC and the total retinal thickness in the 3-mm area at weeks 12 and 24 for BCVA after one year. The overall model accuracy was R2=0.21/0.23 (p<0.001). The outcomes of this pilot analysis highlight the great potential of the proposed machine-learning approach for large-scale image data analysis in DME and other retinal diseases.

Embodied Gesture Processing: Motor-Based Integration of Perception and Action in Social Artificial Agents.

A close coupling of perception and action processes is assumed to play an important role in basic capabilities of social interaction, such as guiding attention and observation of others' behavior, coordinating the form and functions of behavior, or grounding the understanding of others' behavior in one's own experiences. In the attempt to endow artificial embodied agents with similar abilities, we present a probabilistic model for the integration of perception and generation of hand-arm gestures via a hierarchy of shared motor representations, allowing for combined bottom-up and top-down processing. Results from human-agent interactions are reported demonstrating the model's performance in learning, observation, imitation, and generation of gestures.