Joint Multimodal Deep Learning-based Automatic Segmentation of Indocyanine Green Angiography and OCT Images for Assessment of Polypoidal Choroidal Vasculopathy Biomarkers.

Purpose: To develop a fully-automatic hybrid algorithm to jointly segment and quantify biomarkers of polypoidal choroidal vasculopathy (PCV) on indocyanine green angiography (ICGA) and spectral domain-OCT (SD-OCT) images. Design: Evaluation of diagnostic test or technology. Participants: Seventy-two participants with PCV enrolled in clinical studies at Singapore National Eye Center. Methods: The dataset consisted of 2-dimensional (2-D) ICGA and 3-dimensional (3-D) SD-OCT images which were spatially registered and manually segmented by clinicians. A deep learning-based hybrid algorithm called PCV-Net was developed for automatic joint segmentation of biomarkers. The PCV-Net consisted of a 2-D segmentation branch for ICGA and 3-D segmentation branch for SD-OCT. We developed fusion attention modules to connect the 2-D and 3-D branches for effective use of the spatial correspondence between the imaging modalities by sharing learned features. We also used self-supervised pretraining and ensembling to further enhance the performance of the algorithm without the need for additional datasets. We compared the proposed PCV-Net to several alternative model variants. Main Outcome Measures: The PCV-Net was evaluated based on the Dice similarity coefficient (DSC) of the segmentations and the Pearson's correlation and absolute difference of the clinical measurements obtained from the segmentations. Manual grading was used as the gold standard. Results: The PCV-Net showed good performance compared to manual grading and alternative model variants based on both quantitative and qualitative analyses. Compared to the baseline variant, PCV-Net improved the DSC by 0.04 to 0.43 across the different biomarkers, increased the correlations, and decreased the absolute differences of clinical measurements of interest. Specifically, the largest average (mean ± standard error) DSC improvement was for intraretinal fluid, from 0.02 ± 0.00 (baseline variant) to 0.45 ± 0.06 (PCV-Net). In general, improving trends were observed across the model variants as more technical specifications were added, demonstrating the importance of each aspect of the proposed method. Conclusion: The PCV-Net has the potential to aid clinicians in disease assessment and research to improve clinical understanding and management of PCV. Financial Disclosures: Proprietary or commercial disclosure may be found after the references.

Baseline Microperimetry and OCT in the RUSH2A Study: Structure-Function Association and Correlation With Disease Severity.

PURPOSE: To investigate baseline mesopic microperimetry (MP) and spectral domain optical coherence tomography (OCT) in the Rate of Progression in USH2A-related Retinal Degeneration (RUSH2A) study. DESIGN: Natural history study METHODS: Setting: 16 clinical sites in Europe and North AmericaStudy Population: Participants with Usher syndrome type 2 (USH2) (N = 80) or autosomal recessive nonsyndromic RP (ARRP) (N = 47) associated with biallelic disease-causing sequence variants in USH2AObservation Procedures: General linear models were used to assess characteristics including disease duration, MP mean sensitivity and OCT intact ellipsoid zone (EZ) area. The associations between mean sensitivity and EZ area with other measures, including best corrected visual acuity (BCVA) and central subfield thickness (CST) within the central 1 mm, were assessed using Spearman correlation coefficients. MAIN OUTCOME MEASURES: Mean sensitivity on MP; EZ area and CST on OCT. RESULTS: All participants (N = 127) had OCT, while MP was obtained at selected sites (N = 93). Participants with Usher syndrome type 2 (USH2, N = 80) and nonsyndromic autosomal recessive Retinitis Pigmentosa (ARRP, N = 47) had the following similar measurements: EZ area (median (interquartile range [IQR]): 1.4 (0.4, 3.1) mm2 vs 2.3 (0.7, 5.7) mm2) and CST (median (IQR): 247 (223, 280) µm vs 261 (246, 288), and mean sensitivity (median (IQR): 3.5 (2.1, 8.4) dB vs 5.1 (2.9, 9.0) dB). Longer disease duration was associated with smaller EZ area (P < 0.001) and lower mean sensitivity (P = 0.01). Better BCVA, larger EZ area, and larger CST were correlated with greater mean sensitivity (r > 0.3 and P < 0.01). Better BCVA and larger CST were associated with larger EZ area (r > 0.6 and P < 0.001). CONCLUSIONS: Longer disease duration correlated with more severe retinal structure and function abnormalities, and there were associations between MP and OCT metrics. Monitoring changes in retinal structure-function relationships during disease progression will provide important insights into disease mechanism in USH2A-related retinal degeneration.

Impact of Baseline Quantitative OCT Features on Response to Risuteganib for the Treatment of Dry Age-Related Macular Degeneration: The Importance of Outer Retinal Integrity.

OBJECTIVE: The purpose of the study was to perform a post hoc analysis to explore the effect of baseline anatomic characteristics identified on OCT on best-corrected visual acuity (BCVA) responses to risuteganib from the completed phase II study in subjects with dry age-related macular degeneration (AMD). DESIGN: Post hoc analysis of a randomized, double-masked, placebo-controlled, phase II study. SUBJECTS: Eyes with intermediate dry AMD with BCVA between 20/40 and 20/200. Patients with concurrent vision-influencing or macula-obscuring ocular pathologies were excluded. METHODS: Patients were randomized to receive a 1-mg intravitreal risuteganib injection or a sham injection at baseline. A second 1-mg intravitreal injection of risuteganib was given at week 16 to those in the treatment arm. Two independent, masked reading centers evaluated the baseline anatomic characteristics on OCT to explore features associated with positive responses to risuteganib. MAIN OUTCOME MEASURES: Treatment response was defined as a gain of ≥ 8 letters in BCVA from baseline to week 28 in the treatment arm, compared with baseline to week 12 in the sham group. Anatomic parameters, measured by retinal segmentation platforms, including measures of retinal thickness were compared between the responders and nonresponders to risuteganib. RESULTS: Thirty-nine patients completed the study and underwent analysis. In the treatment arm, 48% of eyes demonstrated treatment responses, compared with 7% in the sham group. In the quantitative anatomic assessment, enhanced ellipsoid integrity, greater outer retinal thickness, and decreased geographic atrophy were associated with increased BCVA gains to risuteganib. CONCLUSIONS: This post hoc analysis demonstrated that baseline OCT features may help determine the likelihood of a functional response to risuteganib. The characterization of higher-order OCT features may provide important information regarding biomarkers for treatment response and could facilitate optimized clinical trial enrollment and enrichment.

VALIDATION OF A DEEP LEARNING-BASED ALGORITHM FOR SEGMENTATION OF THE ELLIPSOID ZONE ON OPTICAL COHERENCE TOMOGRAPHY IMAGES OF AN USH2A-RELATED RETINAL DEGENERATION CLINICAL TRIAL.

PURPOSE: To assess the generalizability of a deep learning-based algorithm to segment the ellipsoid zone (EZ). METHODS: The dataset consisted of 127 spectral-domain optical coherence tomography volumes from eyes of participants with USH2A-related retinal degeneration enrolled in the RUSH2A clinical trial (NCT03146078). The EZ was segmented manually by trained readers and automatically by deep OCT atrophy detection, a deep learning-based algorithm originally developed for macular telangiectasia Type 2. Performance was evaluated using the Dice similarity coefficient between the segmentations, and the absolute difference and Pearson's correlation of measurements of interest obtained from the segmentations. RESULTS: With deep OCT atrophy detection, the average (mean ± SD, median) Dice similarity coefficient was 0.79 ± 0.27, 0.90. The average absolute difference in total EZ area was 0.62 ± 1.41, 0.22 mm2 with a correlation of 0.97. The average absolute difference in the maximum EZ length was 222 ± 288, 126 µm with a correlation of 0.97. CONCLUSION: Deep OCT atrophy detection segmented EZ in USH2A-related retinal degeneration with good performance. The algorithm is potentially generalizable to other diseases and other biomarkers of interest as well, which is an important aspect of clinical applicability.

Deep learning-based classification and segmentation of retinal cavitations on optical coherence tomography images of macular telangiectasia type 2.

AIM: To develop a fully automatic algorithm to segment retinal cavitations on optical coherence tomography (OCT) images of macular telangiectasia type 2 (MacTel2). METHODS: The dataset consisted of 99 eyes from 67 participants enrolled in an international, multicentre, phase 2 MacTel2 clinical trial (NCT01949324). Each eye was imaged with spectral-domain OCT at three time points over 2 years. Retinal cavitations were manually segmented by a trained Reader and the retinal cavitation volume was calculated. Two convolutional neural networks (CNNs) were developed that operated in sequential stages. In the first stage, CNN1 classified whether a B-scan contained any retinal cavitations. In the second stage, CNN2 segmented the retinal cavitations in a B-scan. We evaluated the performance of the proposed method against alternative methods using several performance metrics and manual segmentations as the gold standard. RESULTS: The proposed method was computationally efficient and accurately classified and segmented retinal cavitations on OCT images, with a sensitivity of 0.94, specificity of 0.80 and average Dice similarity coefficient of 0.94±0.07 across all time points. The proposed method produced measurements that were highly correlated with the manual measurements of retinal cavitation volume and change in retinal cavitation volume over time. CONCLUSION: The proposed method will be useful to help clinicians quantify retinal cavitations, assess changes over time and further investigate the clinical significance of these early structural changes observed in MacTel2.

Open-Source Automatic Biomarker Measurement on Slit-Lamp Photography to Estimate Visual Acuity in Microbial Keratitis.

Purpose: To assess clinical applicability of automatic image analysis in microbial keratitis (MK) by evaluating the relationship between biomarker measurements on slit-lamp photography (SLP) and best-corrected visual acuity (BCVA). Methods: Seventy-six patients with MK with SLP images and same-day logarithm of the minimum angle of resolution (logMAR) BCVA were evaluated. MK biomarkers (stromal infiltrate, white blood cell infiltration, corneal edema, hypopyon, epithelial defect) were segmented manually by ophthalmologists and automatically by a novel, open-source, deep learning-based segmentation algorithm. Five measurements (presence, maximum width, total area, proportion of the corneal limbus area affected, centrality) were calculated. Correlations between the measurements and BCVA were calculated. An automatic regression model estimated BCVA from the measurements. Differences in performance between using manual and automatic measurements were evaluated using William's test (for correlation) and the paired-sample t-test (for absolute error). Results: Measurements had high correlations of 0.86 (manual) and 0.84 (automatic) with true BCVA. Estimated BCVA had average (mean ± SD) absolute errors of 0.39 ± 0.27 logMAR (manual, median: 0.30) and 0.35 ± 0.28 logMAR (automatic, median: 0.30) and high correlations of 0.76 (manual) and 0.80 (automatic) with true BCVA. Differences between using manual and automatic measurements were not statistically significant for correlations of measurements with true BCVA (P = .66), absolute errors of estimated BCVA (P = .15), or correlations of estimated BCVA with true BCVA (P = .60). Conclusions: The proposed algorithm measured MK biomarkers as accurately as ophthalmologists. Measurements were highly correlated with and estimative of visual acuity. Translational Relevance: This study demonstrates the potential of developing fully automatic objective and standardized strategies to aid ophthalmologists in the clinical assessment of MK.

Intraoperative Retinal Changes May Predict Surgical Outcomes After Epiretinal Membrane Peeling.

Purpose: To investigate whether intraoperative retinal changes during epiretinal membrane (ERM) peeling affect anatomic or functional outcomes after surgery. Methods: We measured retinal thickness using an intraoperative optical coherence tomography (iOCT) device in patients undergoing pars plana vitrectomy with membrane peeling for idiopathic ERM. Changes in intraoperative central macular thickness (iCMT) were compared with postoperative improvements in CMT and best-corrected visual acuity (VA). Results: Twenty-seven eyes from 27 patients (mean age 68 years) underwent iOCT-assisted ERM peeling surgery. Before surgery, mean VA was logMAR 0.50 ± 0.36 (Snellen 20/63), and mean baseline CMT was 489 ± 82 µm. Mean iCMT before peeling was 477 ± 87 µm, which correlated well with preoperative CMT (P < 0.001). Mean change in iCMT was -39.6 ± 37 µm (range -116 to +77 µm). After surgery, VA improved to logMAR 0.40 ± 0.38 (Snellen 20/50) at month 1 and logMAR 0.27 ± 0.23 (Snellen 20/37) at month 3, whereas CMT decreased to 397 ± 44 µm and 396 ± 51 µm at months 1 and 3. Eyes that underwent greater amount of iCMT change (absolute value of iCMT change) were associated with greater CMT reduction at month 1 (P < 0.001) and month 3 (P = 0.010), whereas those with greater intraoperative thinning (actual iCMT change) showed a trend toward better VA outcomes at months 1 (P = 0.054) and 3 (P = 0.036). Conclusions: Intraoperative changes in retinal thickness may predict anatomic and visual outcomes after idiopathic ERM peeling surgery. Translational Relevance: Our study suggests that intraoperative retinal tissue response to ERM peeling surgery measured by iOCT may be a prognostic indicator for restoration of retinal architecture and for visual acuity outcomes.

COMPARISON OF SINGLE DRUSEN SIZE ON COLOR FUNDUS PHOTOGRAPHY AND SPECTRAL-DOMAIN OPTICAL COHERENCE TOMOGRAPHY.

PURPOSE: To determine the relationship of drusen size as determined by spectral-domain optical coherence tomography (SD-OCT) with that measured on registered color fundus photography (CFP) images and to derive an OCT-based classification system that was comparable with that determined by CFP. METHODS: Custom software was developed to register CFP images to the scanning laser ophthalmoscopy fundus images obtained simultaneously with the corresponding SD-OCT images, so that individual drusen observed on CFP could be matched with those seen on SD-OCT. Single druse size (diameter, area, volume, and height) on CFP and SD-OCT images from a phase two clinical trial was determined with the Duke OCT Retinal Analysis Program. RESULTS: The sizes of 213 individual drusen were measured on CFP and SD-OCT. The drusen diameter measured on CFP was significantly correlated with those determined on SD-OCT (R: 0.879, P < 0.001). Based on the corresponding formula: drusen diameter on SD-OCT = 0.77 × (drusen diameter on CFP) + 50.67 µm, large drusen defined as ≥125 µm on CFP had a diameter ≥145 µm on OCT, medium drusen defined as 63 µm to 124 µm on CFP had diameters 100 µm to 144 µm on OCT, and small drusen defined as <63 µm on CFP had diameters <100 µm on OCT. CONCLUSION: With our registration software and imaging processing algorithms, we were able to correlate individual druse sizes measured on CFP with those determined on SD-OCT. These data can be used to develop an SD-OCT-based grading scale, analogous to the CFP Age-Related Eye Disease Study drusen scale that may be useful in the clinic and in clinical trials.

Open-Source Automatic Segmentation of Ocular Structures and Biomarkers of Microbial Keratitis on Slit-Lamp Photography Images Using Deep Learning.

We propose a fully-automatic deep learning-based algorithm for segmentation of ocular structures and microbial keratitis (MK) biomarkers on slit-lamp photography (SLP) images. The dataset consisted of SLP images from 133 eyes with manual annotations by a physician, P1. A modified region-based convolutional neural network, SLIT-Net, was developed and trained using P1's annotations to identify and segment four pathological regions of interest (ROIs) on diffuse white light images (stromal infiltrate (SI), hypopyon, white blood cell (WBC) border, corneal edema border), one pathological ROI on diffuse blue light images (epithelial defect (ED)), and two non-pathological ROIs on all images (corneal limbus, light reflexes). To assess inter-reader variability, 75 eyes were manually annotated for pathological ROIs by a second physician, P2. Performance was evaluated using the Dice similarity coefficient (DSC) and Hausdorff distance (HD). Using seven-fold cross-validation, the DSC of the algorithm (as compared to P1) for all ROIs was good (range: 0.62-0.95) on all 133 eyes. For the subset of 75 eyes with manual annotations by P2, the DSC for pathological ROIs ranged from 0.69-0.85 (SLIT-Net) vs. 0.37-0.92 (P2). DSCs for SLIT-Net were not significantly different than P2 for segmenting hypopyons (p > 0.05) and higher than P2 for WBCs (p < 0.001) and edema (p < 0.001). DSCs were higher for P2 for segmenting SIs (p < 0.001) and EDs (p < 0.001). HDs were lower for P2 for segmenting SIs (p = 0.005) and EDs (p < 0.001) and not significantly different for hypopyons (p > 0.05), WBCs (p > 0.05), and edema (p > 0.05). This prototype fully-automatic algorithm to segment MK biomarkers on SLP images performed to expectations on an exploratory dataset and holds promise for quantification of corneal physiology and pathology.

Measurement Reliability for Keratitis Morphology.

PURPOSE: To evaluate the reliability of manual annotation when quantifying cornea anatomical and microbial keratitis (MK) morphological features on slit-lamp photography (SLP) images. METHODS: Prospectively enrolled patients with MK underwent SLP at initial encounter at 2 academic eye hospitals. Patients who presented with an epithelial defect (ED) were eligible for analysis. Features, which included ED, corneal limbus (L), pupil (P), stromal infiltrate (SI), white blood cell (WBC) infiltration at the SI edge, and hypopyon (H), were annotated independently by 2 physicians on SLP images. Intraclass correlation coefficients (ICCs) were applied for reliability assessment; dice similarity coefficients (DSCs) were used to investigate the area overlap between readers. RESULTS: Seventy-five MK patients with an ED received SLP. DSCs indicate good to fair annotation overlap between graders (L = 0.97, P = 0.80, ED = 0.94, SI = 0.82, H = 0.82, WBC = 0.83) and between repeat annotations by the same grader (L = 0.97, P = 0.81, ED = 0.94, SI = 0.85, H = 0.84, WBC = 0.82). ICC scores showed good intergrader (L = 0.98, P = 0.78, ED = 1.00, SI = 0.67, H = 0.97, WBC = 0.86) and intragrader (L = 0.99, P = 0.92, ED = 0.99, SI = 0.94, H = 0.99, WBC = 0.92) reliabilities. When reliability statistics were recalculated for annotated SI area in the subset of cases where both graders agreed WBC infiltration was present/absent, intergrader ICC improved to 0.91 and DSC improved to 0.86 and intragrader ICC remained the same, whereas DSC improved to 0.87. CONCLUSIONS: Manual annotation indicates usefulness of area quantification in the evaluation of MK. However, variability is intrinsic to the task. Thus, there is a need for optimization of annotation protocols. Future directions may include using multiple annotators per image or automated annotation software.

Beyond Performance Metrics: Automatic Deep Learning Retinal OCT Analysis Reproduces Clinical Trial Outcome.

PURPOSE: To validate the efficacy of a fully automatic, deep learning-based segmentation algorithm beyond conventional performance metrics by measuring the primary outcome of a clinical trial for macular telangiectasia type 2 (MacTel2). DESIGN: Evaluation of diagnostic test or technology. PARTICIPANTS: A total of 92 eyes from 62 participants with MacTel2 from a phase 2 clinical trial (NCT01949324) randomized to 1 of 2 treatment groups METHODS: The ellipsoid zone (EZ) defect areas were measured on spectral domain OCT images of each eye at 2 time points (baseline and month 24) by a fully automatic, deep learning-based segmentation algorithm. The change in EZ defect area from baseline to month 24 was calculated and analyzed according to the clinical trial protocol. MAIN OUTCOME MEASURE: Difference in the change in EZ defect area from baseline to month 24 between the 2 treatment groups. RESULTS: The difference in the change in EZ defect area from baseline to month 24 between the 2 treatment groups measured by the fully automatic segmentation algorithm was 0.072±0.035 mm2 (P = 0.021). This was comparable to the outcome of the clinical trial using semiautomatic measurements by expert readers, 0.065±0.033 mm2 (P = 0.025). CONCLUSIONS: The fully automatic segmentation algorithm was as accurate as semiautomatic expert segmentation to assess EZ defect areas and was able to reliably reproduce the statistically significant primary outcome measure of the clinical trial. This approach, to validate the performance of an automatic segmentation algorithm on the primary clinical trial end point, provides a robust gauge of its clinical applicability.

Computational modeling of retinal hypoxia and photoreceptor degeneration in patients with age-related macular degeneration.

Although drusen have long been acknowledged as a primary hallmark of dry age-related macular degeneration (AMD) their role in the disease remains unclear. We hypothesize that drusen accumulation increases the barrier to metabolite transport ultimately resulting in photoreceptor cell death. To investigate this hypothesis, a computational model was developed to evaluate steady-state oxygen distribution in the retina. Optical coherence tomography images from fifteen AMD patients and six control subjects were segmented and translated into 3D in silico representations of retinal morphology. A finite element model was then used to determine the steady-state oxygen distribution throughout the retina for both generic and patient-specific retinal morphology. Oxygen levels were compared to the change in retinal thickness at a later time point to observe possible correlations. The generic finite element model of oxygen concentration in the retina agreed closely with both experimental measurements from literature and clinical observations, including the minimal pathological drusen size identified by AREDS (64 µm). Modeling oxygen distribution in the outer retina of AMD patients showed a substantially stronger correlation between hypoxia and future retinal thinning (Pearson correlation coefficient, r = 0.2162) than between drusen height and retinal thinning (r = 0.0303) indicating the potential value of this physiology-based approach. This study presents proof-of-concept for the potential utility of finite element modeling in evaluating retinal health and also suggests a potential link between transport and AMD pathogenesis. This strategy may prove useful as a prognostic tool for predicting the clinical risk of AMD progression.

Deep longitudinal transfer learning-based automatic segmentation of photoreceptor ellipsoid zone defects on optical coherence tomography images of macular telangiectasia type 2.

Photoreceptor ellipsoid zone (EZ) defects visible on optical coherence tomography (OCT) are important imaging biomarkers for the onset and progression of macular diseases. As such, accurate quantification of EZ defects is paramount to monitor disease progression and treatment efficacy over time. We developed and trained a novel deep learning-based method called Deep OCT Atrophy Detection (DOCTAD) to automatically segment EZ defect areas by classifying 3-dimensional A-scan clusters as normal or defective. Furthermore, we introduce a longitudinal transfer learning paradigm in which the algorithm learns from segmentation errors on images obtained at one time point to segment subsequent images with higher accuracy. We evaluated the performance of this method on 134 eyes of 67 subjects enrolled in a clinical trial of a novel macular telangiectasia type 2 (MacTel2) therapeutic agent. Our method compared favorably to other deep learning-based and non-deep learning-based methods in matching expert manual segmentations. To the best of our knowledge, this is the first automatic segmentation method developed for EZ defects on OCT images of MacTel2.

Modeling the biomechanics of fetal movements.

Fetal movements in the uterus are a natural part of development and are known to play an important role in normal musculoskeletal development. However, very little is known about the biomechanical stimuli that arise during movements in utero, despite these stimuli being crucial to normal bone and joint formation. Therefore, the objective of this study was to create a series of computational steps by which the forces generated during a kick in utero could be predicted from clinically observed fetal movements using novel cine-MRI data of three fetuses, aged 20-22 weeks. A custom tracking software was designed to characterize the movements of joints in utero, and average uterus deflection of [Formula: see text] mm due to kicking was calculated. These observed displacements provided boundary conditions for a finite element model of the uterine environment, predicting an average reaction force of [Formula: see text] N generated by a kick against the uterine wall. Finally, these data were applied as inputs for a musculoskeletal model of a fetal kick, resulting in predicted maximum forces in the muscles surrounding the hip joint of approximately 8 N, while higher maximum forces of approximately 21 N were predicted for the muscles surrounding the knee joint. This study provides a novel insight into the closed mechanical environment of the uterus, with an innovative method allowing elucidation of the biomechanical interaction of the developing fetus with its surroundings.