Panretinal handheld OCT angiography for pediatric retinal imaging.

Comprehensive visualization of retina morphology is essential in the diagnosis and management of retinal diseases in pediatric populations. Conventional imaging techniques often face challenges in effectively capturing the peripheral retina, primarily due to the limitations in current optical designs, which lack the necessary field of view to characterize the far periphery. To address this gap, our study introduces a novel ultra-widefield optical coherence tomography angiography (OCTA) system. This system, specifically tailored for pediatric applications, incorporates an ultrahigh-speed 800 kHz swept-source laser. The system's innovative design achieves a 140° field of view while maintaining excellent optical performance. Over the last 15 months, we have conducted 379 eye examinations on 96 babies using this system. It demonstrates marked efficacy in the diagnosis of retinopathy of prematurity, providing detailed and comprehensive peripheral retinal angiography. The capabilities of the ultra-widefield handheld OCTA system in enhancing the clarity and thoroughness of retina vascularization assessments have significantly improved the precision of diagnoses and the customization of treatment strategies. Our findings underscore the system's potential to advance pediatric ophthalmology and broaden the scope of retinal imaging.

Use of an artificial intelligence-generated vascular severity score improved plus disease diagnosis in retinopathy of prematurity.

OBJECTIVE: To evaluate whether providing clinicians with an artificial intelligence-based vascular severity score (AI-VSS) improves consistency in diagnosis of plus disease in retinopathy of prematurity (ROP). DESIGN: This is a multi-reader diagnostic accuracy imaging study. PARTICIPANTS: Eleven ROP experts (4 pediatric ophthalmologists, 7 retina specialists), 9 of which had been in practice for 10 or more years. METHODS: Retcam (Natus Medical Incorporated) fundus images were obtained from premature infants during routine ROP screening as part of the Imaging and Informatics in ROP study between January 2012 and July 2020. From all available exams, a subset of 150 eye exams from 110 infants were selected for grading. An AI-VSS was assigned to each set of images using the i-ROP DL system. The clinicians were asked to diagnose plus disease for each exam and assign an estimated VSS (range 1-9) at baseline, and then again one month later with AI-VSS assistance. A reference standard diagnosis (RSD) was assigned to each eye exam from the i-ROP study based on 3 masked expert labels and the ophthalmoscopic diagnosis. MAIN OUTCOME MEASURE: Mean linearly weighted kappa for plus disease diagnosis compared to the RSD. Area under the receiver operating characteristic and precision-recall curves (AUROC, AUPR) for 1-9 labels compared to RSD for plus disease. RESULTS: Expert agreement improved significantly from substantial (κ: 0.69 [0.59, 0.75]) to near perfect (κ: 0.81 [0.71, 0.86]) when AI-VSS was integrated. Additionally, there was a significant improvement in plus disease discrimination as measured by mean [95% confidence interval] AUROC (0.94 [0.92, 0.96] to 0.98 [0.96, 0.99], difference: 0.04 [0.01, 0.06]) and AUPR (0.86 [0.81, 0.90] to 0.95 [0.91, 0.97], difference: 0.09 [0.03, 0.14]). CONCLUSIONS: Providing ROP clinicians with an AI-based measurement of vascular severity in ROP was associated with both improved plus disease diagnosis and improved continuous severity labeling, as compared to a reference standard diagnosis for plus disease. If implemented in practice, AI-VSS could reduce inter-observer variability and standardize treatment for infants with ROP.

P-Score: A Reference-Image-Based Clinical Grading Scale for Vascular Change in Retinopathy of Prematurity.

PURPOSE: The International Classification of Retinopathy of Prematurity Third Edition (ICROP3) acknowledged that plus-like ROP vascular changes occur along a spectrum. Historically, clinician-experts demonstrate variable agreement for plus diagnosis. We developed a 9-photo reference-image set for grading plus-like changes and compared intergrader agreement of the set to standard grading with no-plus/pre-plus/plus. DESIGN: Retinal photographic grading and expert consensus opinion PARTICIPANTS: Development: 34 international ICROP3 committee members. VALIDATION: 30 ophthalmologists with ROP expertise (15 ICROP3 committee members, 15 non-ICROP3 members) METHODS: Nine ROP fundus images (P1 through P9) representing increasing degrees of zone I vascular tortuosity and dilation, based on ICROP3-committee's 34 members' gradings and consensus image review, were used to establish standard photographs for the "Plus (P) Score." Study participants graded 150 fundus photographs two ways, separated by a 1-week washout period: (1) no-plus/pre-plus/plus disease, (2) choosing the closest P-Score image. MAIN OUTCOME MEASURES: Intergrader agreement measured by intraclass correlation coefficient (ICC) RESULTS: Intergrader agreement was higher using P-Score (ICC 0.75, 95% CI 0.71-0.79) than no-plus/pre-plus/plus (ICC 0.67, 95% CI 0.62-0.72). Mean P-Scores for images whose mode gradings were no-plus, pre-plus, and plus, were 2.5 (SD 0.7), 4.8 (SD 0.8), and 7.4 (SD 0.8), respectively. CONCLUSIONS: Intergrader agreement of plus-like vascular change in ROP using the P-Score is high. We recommend incorporation of this 9-image reference set into ICROP3 and clinician daily practice alongside zone/stage/plus. P-score is not yet meant to replace plus diagnosis for treatment decisions, but its use at our institutions has permitted better comparison between examinations for progression and regression, communication between examiners, and documentation of vascular change without fundus imaging. P-score also could provide more detailed ROP classification for clinical trials, consistent with the spectrum of plus-like change that is now formally part of ICROP.

Isolated Retinal Neovascularization in Retinopathy of Prematurity: Clinical Associations and Prognostic Implications.

OBJECTIVE: Isolated retinal neovascularization (IRNV) is a common finding in patients with stage 2 and 3 retinopathy of prematurity (ROP). This study aimed to further classify the clinical course and significance of these lesions (previously described as "popcorn" based on clinical appearance) in patients with ROP as visualized with ultrawidefield OCT (UWF-OCT). DESIGN: Single center, retrospective case series. PARTICIPANTS: Images were collected from 136 babies in the Oregon Health and Science University neonatal intensive care unit. METHODS: A prototype UWF-OCT device captured en face scans (>140°), which were reviewed for the presence of IRNV along with standard zone, stage, and plus classification. In a cross-sectional analysis we compared demographics and the clinical course of eyes with and without IRNV. Longitudinally, we compared ROP severity using a clinician-assigned vascular severity score (VSS) and compared the risk of progression among eyes with and without IRNV using multivariable logistic regression. MAIN OUTCOME MEASURES: Differences in clinical demographics and disease progression between patients with and without IRNV. RESULTS: Of the 136 patients, 60 developed stage 2 or worse ROP during their disease course, 22 of whom had IRNV visualized on UWF-OCT (37%). On average, patients with IRNV had lower birth weights (BWs) (660.1 vs. 916.8 g, P = 0.001), gestational age (GA) (24.9 vs. 26.1 weeks, P = 0.01), and were more likely to present with ROP in zone I (63.4% vs. 15.8%, P < 0.001). They were also more likely to progress to stage 3 (68.2% vs. 13.2%, P < 0.001) and receive treatment (54.5% vs. 15.8%, P = 0.002). Eyes with IRNV had a higher peak VSS (5.61 vs. 3.73, P < 0.001) and averaged a higher VSS throughout their disease course. On multivariable logistic regression, IRNV was independently associated with progression to stage 3 (P = 0.02) and requiring treatment (P = 0.03), controlling for GA, BW, and initial zone 1 disease. CONCLUSIONS: In this single center study, we found that IRNV occurs in higher risk babies and was an independent risk factor for ROP progression and treatment. These findings may have implications for OCT-based ROP classifications in the future. FINANCIAL DISCLOSURE(S): Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Multinational External Validation of Autonomous Retinopathy of Prematurity Screening.

Importance: Retinopathy of prematurity (ROP) is a leading cause of blindness in children, with significant disparities in outcomes between high-income and low-income countries, due in part to insufficient access to ROP screening. Objective: To evaluate how well autonomous artificial intelligence (AI)-based ROP screening can detect more-than-mild ROP (mtmROP) and type 1 ROP. Design, Setting, and Participants: This diagnostic study evaluated the performance of an AI algorithm, trained and calibrated using 2530 examinations from 843 infants in the Imaging and Informatics in Retinopathy of Prematurity (i-ROP) study, on 2 external datasets (6245 examinations from 1545 infants in the Stanford University Network for Diagnosis of ROP [SUNDROP] and 5635 examinations from 2699 infants in the Aravind Eye Care Systems [AECS] telemedicine programs). Data were taken from 11 and 48 neonatal care units in the US and India, respectively. Data were collected from January 2012 to July 2021, and data were analyzed from July to December 2023. Exposures: An imaging processing pipeline was created using deep learning to autonomously identify mtmROP and type 1 ROP in eye examinations performed via telemedicine. Main Outcomes and Measures: The area under the receiver operating characteristics curve (AUROC) as well as sensitivity and specificity for detection of mtmROP and type 1 ROP at the eye examination and patient levels. Results: The prevalence of mtmROP and type 1 ROP were 5.9% (91 of 1545) and 1.2% (18 of 1545), respectively, in the SUNDROP dataset and 6.2% (168 of 2699) and 2.5% (68 of 2699) in the AECS dataset. Examination-level AUROCs for mtmROP and type 1 ROP were 0.896 and 0.985, respectively, in the SUNDROP dataset and 0.920 and 0.982 in the AECS dataset. At the cross-sectional examination level, mtmROP detection had high sensitivity (SUNDROP: mtmROP, 83.5%; 95% CI, 76.6-87.7; type 1 ROP, 82.2%; 95% CI, 81.2-83.1; AECS: mtmROP, 80.8%; 95% CI, 76.2-84.9; type 1 ROP, 87.8%; 95% CI, 86.8-88.7). At the patient level, all infants who developed type 1 ROP screened positive (SUNDROP: 100%; 95% CI, 81.4-100; AECS: 100%; 95% CI, 94.7-100) prior to diagnosis. Conclusions and Relevance: Where and when ROP telemedicine programs can be implemented, autonomous ROP screening may be an effective force multiplier for secondary prevention of ROP.

Genome-wide association identifies novel ROP risk loci in a multiethnic cohort.

We conducted a genome-wide association study (GWAS) in a multiethnic cohort of 920 at-risk infants for retinopathy of prematurity (ROP), a major cause of childhood blindness, identifying 1 locus at genome-wide significance level (p < 5×10-8) and 9 with significance of p < 5×10-6 for ROP ≥ stage 3. The most significant locus, rs2058019, reached genome-wide significance within the full multiethnic cohort (p = 4.96×10-9); Hispanic and European Ancestry infants driving the association. The lead single nucleotide polymorphism (SNP) falls in an intronic region within the Glioma-associated oncogene family zinc finger 3 (GLI3) gene. Relevance for GLI3 and other top-associated genes to human ocular disease was substantiated through in-silico extension analyses, genetic risk score analysis and expression profiling in human donor eye tissues. Thus, we identify a novel locus at GLI3 with relevance to retinal biology, supporting genetic susceptibilities for ROP risk with possible variability by race and ethnicity.

Oxygenation Fluctuations Associated with Severe Retinopathy of Prematurity: Insights from a Multimodal Deep Learning Approach.

Purpose: Retinopathy of prematurity (ROP) is one of the leading causes of blindness in children. Although the role of oxygen in the pathophysiology of ROP is well established, a precise understanding of the dynamic relationship between oxygen exposure ROP incidence and severity is lacking. The purpose of this study was to evaluate the correlation between time-dependent oxygen variables and the onset of ROP. Design: Retrospective cohort study. Participants: Two hundred thirty infants who were born at a single academic center and met the inclusion criteria were included. Infants are mainly born between January 2011 and October 2022. Methods: Patient data were extracted from electronic health records (EHRs), with sufficient time-dependent oxygen data. Clinical outcomes for ROP were recorded as none/mild or moderate/severe (defined as type II or worse). Mixed-effects linear models were used to compare the 2 groups in terms of dynamic oxygen variables, such as daily average and the coefficient of variation (COV) fraction of inspired oxygen (FiO2). Support vector machine (SVM) and long-short-term memory (LSTM)-based multimodal models were trained with fivefold cross-validation to predict which infants would develop moderate/severe ROP. Gestational age (GA), birth weight, and time-dependent oxygen variables were used to develop predictive models. Main Outcome Measures: Model cross-validation performance was evaluated by computing the mean area under the receiver operating characteristic (AUROC) curve, precision, recall, and F1 score. Results: We found that both daily average and COV of FiO2 were associated with more severe ROP (adjusted P < 0.001). With fivefold cross-validation, the multimodal LSTM models had higher performance than the best static models (SVM using GA and 3 average FiO2 features) and SVM models trained on GA alone (mean AUROC = 0.89 ± 0.04 vs. 0.86 ± 0.05 vs. 0.83 ± 0.04). Conclusions: The development of severe ROP might not only be influenced by oxygen exposure but also by its fluctuation, which provides direction for future study of pathophysiological factors associated with severe ROP development. Additionally, we demonstrated that multimodal neural networks can be a method to extract useful information from time-series data, which may be a valuable methodology for the investigation of other diseases using EHR data. Financial Disclosures: Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Genome-wide association identifies novel ROP risk loci in a multi-ethnic cohort.

We conducted a genome-wide association study (GWAS) in a multiethnic cohort of 920 at-risk infants for retinopathy of prematurity (ROP), a major cause of childhood blindness, identifying 2 loci at genome-wide significance level (p<5×10-8) and 7 at suggestive significance (p<5×10-6) for ROP ≥ stage 3. The most significant locus, rs2058019, reached genome-wide significance within the full multiethnic cohort (p=4.96×10-9); Hispanic and Caucasian infants driving the association. The lead single nucleotide polymorphism (SNP) falls in an intronic region within the Glioma-associated oncogene family zinc finger 3 (GLI3) gene. Relevance for GLI3 and other top-associated genes to human ocular disease was substantiated through in-silico extension analyses, genetic risk score analysis and expression profiling in human donor eye tissues. Thus, we report the largest ROP GWAS to date, identifying a novel locus at GLI3 with relevance to retinal biology supporting genetic susceptibilities for ROP risk with possible variability by race and ethnicity.

Association of Biomarker-Based Artificial Intelligence With Risk of Racial Bias in Retinal Images.

Importance: Although race is a social construct, it is associated with variations in skin and retinal pigmentation. Image-based medical artificial intelligence (AI) algorithms that use images of these organs have the potential to learn features associated with self-reported race (SRR), which increases the risk of racially biased performance in diagnostic tasks; understanding whether this information can be removed, without affecting the performance of AI algorithms, is critical in reducing the risk of racial bias in medical AI. Objective: To evaluate whether converting color fundus photographs to retinal vessel maps (RVMs) of infants screened for retinopathy of prematurity (ROP) removes the risk for racial bias. Design, Setting, and Participants: The retinal fundus images (RFIs) of neonates with parent-reported Black or White race were collected for this study. A u-net, a convolutional neural network (CNN) that provides precise segmentation for biomedical images, was used to segment the major arteries and veins in RFIs into grayscale RVMs, which were subsequently thresholded, binarized, and/or skeletonized. CNNs were trained with patients' SRR labels on color RFIs, raw RVMs, and thresholded, binarized, or skeletonized RVMs. Study data were analyzed from July 1 to September 28, 2021. Main Outcomes and Measures: Area under the precision-recall curve (AUC-PR) and area under the receiver operating characteristic curve (AUROC) at both the image and eye level for classification of SRR. Results: A total of 4095 RFIs were collected from 245 neonates with parent-reported Black (94 [38.4%]; mean [SD] age, 27.2 [2.3] weeks; 55 majority sex [58.5%]) or White (151 [61.6%]; mean [SD] age, 27.6 [2.3] weeks, 80 majority sex [53.0%]) race. CNNs inferred SRR from RFIs nearly perfectly (image-level AUC-PR, 0.999; 95% CI, 0.999-1.000; infant-level AUC-PR, 1.000; 95% CI, 0.999-1.000). Raw RVMs were nearly as informative as color RFIs (image-level AUC-PR, 0.938; 95% CI, 0.926-0.950; infant-level AUC-PR, 0.995; 95% CI, 0.992-0.998). Ultimately, CNNs were able to learn whether RFIs or RVMs were from Black or White infants regardless of whether images contained color, vessel segmentation brightness differences were nullified, or vessel segmentation widths were uniform. Conclusions and Relevance: Results of this diagnostic study suggest that it can be very challenging to remove information relevant to SRR from fundus photographs. As a result, AI algorithms trained on fundus photographs have the potential for biased performance in practice, even if based on biomarkers rather than raw images. Regardless of the methodology used for training AI, evaluating performance in relevant subpopulations is critical.

Efficacy of Smartphone-Based Telescreening for Retinopathy of Prematurity With and Without Artificial Intelligence in India.

Importance: Retinopathy of prematurity (ROP) telemedicine screening programs have been found to be effective, but they rely on widefield digital fundus imaging (WDFI) cameras, which are expensive, making them less accessible in low- to middle-income countries. Cheaper, smartphone-based fundus imaging (SBFI) systems have been described, but these have a narrower field of view (FOV) and have not been tested in a real-world, operational telemedicine setting. Objective: To assess the efficacy of SBFI systems compared with WDFI when used by technicians for ROP screening with both artificial intelligence (AI) and human graders. Design, Setting, and Participants: This prospective cross-sectional comparison study took place as a single-center ROP teleophthalmology program in India from January 2021 to April 2022. Premature infants who met normal ROP screening criteria and enrolled in the teleophthalmology screening program were included. Those who had already been treated for ROP were excluded. Exposures: All participants had WDFI images and from 1 of 2 SBFI devices, the Make-In-India (MII) Retcam or Keeler Monocular Indirect Ophthalmoscope (MIO) devices. Two masked readers evaluated zone, stage, plus, and vascular severity scores (VSS, from 1-9) in all images. Smartphone images were then stratified by patient into training (70%), validation (10%), and test (20%) data sets and used to train a ResNet18 deep learning architecture for binary classification of normal vs preplus or plus disease, which was then used for patient-level predictions of referral warranted (RW)- and treatment requiring (TR)-ROP. Main Outcome and Measures: Sensitivity and specificity of detection of RW-ROP, and TR-ROP by both human graders and an AI system and area under the receiver operating characteristic curve (AUC) of grader-assigned VSS. Sensitivity and specificity were compared between the 2 SBFI systems using Pearson χ2testing. Results: A total of 156 infants (312 eyes; mean [SD] gestational age, 33.0 [3.0] weeks; 75 [48%] female) were included with paired examinations. Sensitivity and specificity were not found to be statistically different between the 2 SBFI systems. Human graders were effective with SBFI at detecting TR-ROP with a sensitivity of 100% and specificity of 83.49%. The AUCs with grader-assigned VSS only were 0.95 (95% CI, 0.91-0.99) and 0.96 (95% CI, 0.93-0.99) for RW-ROP and TR-ROP, respectively. For the AI system, the sensitivity of detecting TR-ROP sensitivity was 100% with specificity of 58.6%, and RW-ROP sensitivity was 80.0% with specificity of 59.3%. Conclusions and Relevance: In this cross-sectional study, 2 different SBFI systems used by technicians in an ROP screening program were highly sensitive for TR-ROP. SBFI systems with AI may be a cost-effective method to improve the global capacity for ROP screening.

Panretinal Optical Coherence Tomography.

We introduce a new concept of panoramic retinal (panretinal) optical coherence tomography (OCT) imaging system with a 140° field of view (FOV). To achieve this unprecedented FOV, a contact imaging approach was used which enabled faster, more efficient, and quantitative retinal imaging with measurement of axial eye length. The utilization of the handheld panretinal OCT imaging system could allow earlier recognition of peripheral retinal disease and prevent permanent vision loss. In addition, adequate visualization of the peripheral retina has a great potential for better understanding disease mechanisms regarding the periphery. To the best of our knowledge, the panretinal OCT imaging system presented in this manuscript has the widest FOV among all the retina OCT imaging systems and offers significant values in both clinical ophthalmology and basic vision science.

Development and international validation of custom-engineered and code-free deep-learning models for detection of plus disease in retinopathy of prematurity: a retrospective study.

BACKGROUND: Retinopathy of prematurity (ROP), a leading cause of childhood blindness, is diagnosed through interval screening by paediatric ophthalmologists. However, improved survival of premature neonates coupled with a scarcity of available experts has raised concerns about the sustainability of this approach. We aimed to develop bespoke and code-free deep learning-based classifiers for plus disease, a hallmark of ROP, in an ethnically diverse population in London, UK, and externally validate them in ethnically, geographically, and socioeconomically diverse populations in four countries and three continents. Code-free deep learning is not reliant on the availability of expertly trained data scientists, thus being of particular potential benefit for low resource health-care settings. METHODS: This retrospective cohort study used retinal images from 1370 neonates admitted to a neonatal unit at Homerton University Hospital NHS Foundation Trust, London, UK, between 2008 and 2018. Images were acquired using a Retcam Version 2 device (Natus Medical, Pleasanton, CA, USA) on all babies who were either born at less than 32 weeks gestational age or had a birthweight of less than 1501 g. Each images was graded by two junior ophthalmologists with disagreements adjudicated by a senior paediatric ophthalmologist. Bespoke and code-free deep learning models (CFDL) were developed for the discrimination of healthy, pre-plus disease, and plus disease. Performance was assessed internally on 200 images with the majority vote of three senior paediatric ophthalmologists as the reference standard. External validation was on 338 retinal images from four separate datasets from the USA, Brazil, and Egypt with images derived from Retcam and the 3nethra neo device (Forus Health, Bengaluru, India). FINDINGS: Of the 7414 retinal images in the original dataset, 6141 images were used in the final development dataset. For the discrimination of healthy versus pre-plus or plus disease, the bespoke model had an area under the curve (AUC) of 0·986 (95% CI 0·973-0·996) and the CFDL model had an AUC of 0·989 (0·979-0·997) on the internal test set. Both models generalised well to external validation test sets acquired using the Retcam for discriminating healthy from pre-plus or plus disease (bespoke range was 0·975-1·000 and CFDL range was 0·969-0·995). The CFDL model was inferior to the bespoke model on discriminating pre-plus disease from healthy or plus disease in the USA dataset (CFDL 0·808 [95% CI 0·671-0·909, bespoke 0·942 [0·892-0·982]], p=0·0070). Performance also reduced when tested on the 3nethra neo imaging device (CFDL 0·865 [0·742-0·965] and bespoke 0·891 [0·783-0·977]). INTERPRETATION: Both bespoke and CFDL models conferred similar performance to senior paediatric ophthalmologists for discriminating healthy retinal images from ones with features of pre-plus or plus disease; however, CFDL models might generalise less well when considering minority classes. Care should be taken when testing on data acquired using alternative imaging devices from that used for the development dataset. Our study justifies further validation of plus disease classifiers in ROP screening and supports a potential role for code-free approaches to help prevent blindness in vulnerable neonates. FUNDING: National Institute for Health Research Biomedical Research Centre based at Moorfields Eye Hospital NHS Foundation Trust and the University College London Institute of Ophthalmology. TRANSLATIONS: For the Portuguese and Arabic translations of the abstract see Supplementary Materials section.

Assessment of Retinopathy of Prematurity Regression and Reactivation Using an Artificial Intelligence-Based Vascular Severity Score.

Importance: One of the biggest challenges when using anti-vascular endothelial growth factor (VEGF) agents to treat retinopathy of prematurity (ROP) is the need to perform long-term follow-up examinations to identify eyes at risk of ROP reactivation requiring retreatment. Objective: To evaluate whether an artificial intelligence (AI)-based vascular severity score (VSS) can be used to analyze ROP regression and reactivation after anti-VEGF treatment and potentially identify eyes at risk of ROP reactivation requiring retreatment. Design, Setting, and Participants: This prognostic study was a secondary analysis of posterior pole fundus images collected during the multicenter, double-blind, investigator-initiated Comparing Alternative Ranibizumab Dosages for Safety and Efficacy in Retinopathy of Prematurity (CARE-ROP) randomized clinical trial, which compared 2 different doses of ranibizumab (0.12 mg vs 0.20 mg) for the treatment of ROP. The CARE-ROP trial screened and enrolled infants between September 5, 2014, and July 14, 2016. A total of 1046 wide-angle fundus images obtained from 19 infants at predefined study time points were analyzed. The analyses of VSS were performed between January 20, 2021, and November 18, 2022. Interventions: An AI-based algorithm assigned a VSS between 1 (normal) and 9 (most severe) to fundus images. Main Outcomes and Measures: Analysis of VSS in infants with ROP over time and VSS comparisons between the 2 treatment groups (0.12 mg vs 0.20 mg of ranibizumab) and between infants who did and did not receive retreatment for ROP reactivation. Results: Among 19 infants with ROP in the CARE-ROP randomized clinical trial, the median (range) postmenstrual age at first treatment was 36.4 (34.7-39.7) weeks; 10 infants (52.6%) were male, and 18 (94.7%) were White. The mean (SD) VSS was 6.7 (1.9) at baseline and significantly decreased to 2.7 (1.9) at week 1 (P < .001) and 2.9 (1.3) at week 4 (P < .001). The mean (SD) VSS of infants with ROP reactivation requiring retreatment was 6.5 (1.9) at the time of retreatment, which was significantly higher than the VSS at week 4 (P < .001). No significant difference was found in VSS between the 2 treatment groups, but the change in VSS between baseline and week 1 was higher for infants who later required retreatment (mean [SD], 7.8 [1.3] at baseline vs 1.7 [0.7] at week 1) vs infants who did not (mean [SD], 6.4 [1.9] at baseline vs 3.0 [2.0] at week 1). In eyes requiring retreatment, higher baseline VSS was correlated with earlier time of retreatment (Pearson r = -0.9997; P < .001). Conclusions and Relevance: In this study, VSS decreased after ranibizumab treatment, consistent with clinical disease regression. In cases of ROP reactivation requiring retreatment, VSS increased again to values comparable with baseline values. In addition, a greater change in VSS during the first week after initial treatment was found to be associated with a higher risk of later ROP reactivation, and high baseline VSS was correlated with earlier retreatment. These findings may have implications for monitoring ROP regression and reactivation after anti-VEGF treatment.

Characterization of Errors in Retinopathy of Prematurity Diagnosis by Ophthalmologists-in-Training in Middle-Income Countries.

PURPOSE: To characterize common errors in the diagnosis of retinopathy of prematurity (ROP) among ophthalmologistsin-training in middle-income countries. METHODS: In this prospective cohort study, 200 ophthalmologists-in-training from programs in Brazil, Mexico, and the Philippines participated. A secure web-based educational system was developed using a repository of more than 2,500 unique image sets of ROP, and a reference standard diagnosis was established by combining the clinical diagnosis and the image-based diagnosis by multiple experts. Twenty web-based cases of wide-field retinal images were presented, and ophthalmologists-in-training were asked to diagnose plus disease, zone, stage, and category for each eye. Trainees' responses were compared to the consensus reference standard diagnosis. Main outcome measures were frequency and types of diagnostic errors were analyzed. RESULTS: The error rate in the diagnosis of any category of ROP was between 48% and 59% for all countries. The error rate in identifying type 2 or pre-plus disease was 77%, with a tendency for overdiagnosis (27% underdiagnosis vs 50% overdiagnosis; mean difference: 23.4; 95% CI: 12.1 to 34.7; P = .005). Misdiagnosis of treatment-requiring ROP as type 2 ROP was most commonly associated with incorrectly identifying plus disease (plus disease error rate = 18% with correct category diagnosis vs 69% when misdiagnosed; mean difference: 51.0; 95% CI: 49.3 to 52.7; P = .003). CONCLUSIONS: Ophthalmologists-in-training from middle-income countries misdiagnosed ROP more than half of the time. Identification of plus disease was the salient factor leading to incorrect diagnosis. These findings emphasize the need for improved access to ROP education to improve competency in diagnosis among ophthalmologists-in-training in middle-income countries. [J Pediatr Ophthalmol Strabismus. 2023;60(5):344-352.].

Characterization of Errors in Retinopathy of Prematurity Diagnosis by Ophthalmologists-in-Training in the United States and Canada.

PURPOSE: To identify the prominent factors that lead to misdiagnosis of retinopathy of prematurity (ROP) by ophthalmologists-in-training in the United States and Canada. METHODS: This prospective cohort study included 32 ophthalmologists-in-training at six ophthalmology training programs in the United States and Canada. Twenty web-based cases of ROP using wide-field retinal images were presented, and ophthalmologists-in-training were asked to diagnose plus disease, zone, stage, and category for each eye. Responses were compared to a consensus reference standard diagnosis for accuracy, which was established by combining the clinical diagnosis and the image-based diagnosis by multiple experts. The types of diagnostic errors that occurred were analyzed with descriptive and chi-squared analysis. Main outcome measures were frequency of types (category, zone, stage, plus disease) of diagnostic errors; association of errors in zone, stage, and plus disease diagnosis with incorrectly identified category; and performance of ophthalmologists-in-training across postgraduate years. RESULTS: Category of ROP was misdiagnosed at a rate of 48%. Errors in classification of plus disease were most commonly associated with misdiagnosis of treatment-requiring (plus error rate = 16% when treatment-requiring was correctly diagnosed vs 81% when underdiagnosed as type 2 or pre-plus; mean difference: 64.3; 95% CI: 51.9 to 76.7; P < .001) and type 2 or pre-plus (plus error rate = 35% when type 2 or pre-plus was correctly diagnosed vs 76% when overdiagnosed as treatment-requiring; mean difference: 41.0; 95% CI: 28.4 to 53.5; P < .001) disease. The diagnostic error rate of postgraduate year (PGY)-2 trainees was significantly higher than PGY-3 trainees (PGY-2 category error rate = 61% vs PGY-3 = 35%; mean difference, 25.4; 95% CI: 17.7 to 33.0; P < .001). CONCLUSIONS: Ophthalmologists-in-training in the United States and Canada misdiagnosed ROP nearly half of the time, with incorrect identification of plus disease as a leading cause. Integration of structured learning for ROP in residency education may improve diagnostic competency. [J Pediatr Ophthalmol Strabismus. 2023;60(5):337-343.].

Evaluation of an Artificial Intelligence System for Retinopathy of Prematurity Screening in Nepal and Mongolia.

Purpose: To evaluate the performance of a deep learning (DL) algorithm for retinopathy of prematurity (ROP) screening in Nepal and Mongolia. Design: Retrospective analysis of prospectively collected clinical data. Participants: Clinical information and fundus images were obtained from infants in 2 ROP screening programs in Nepal and Mongolia. Methods: Fundus images were obtained using the Forus 3nethra neo (Forus Health) in Nepal and the RetCam Portable (Natus Medical, Inc.) in Mongolia. The overall severity of ROP was determined from the medical record using the International Classification of ROP (ICROP). The presence of plus disease was determined independently in each image using a reference standard diagnosis. The Imaging and Informatics for ROP (i-ROP) DL algorithm was trained on images from the RetCam to classify plus disease and to assign a vascular severity score (VSS) from 1 through 9. Main Outcome Measures: Area under the receiver operating characteristic curve and area under the precision-recall curve for the presence of plus disease or type 1 ROP and association between VSS and ICROP disease category. Results: The prevalence of type 1 ROP was found to be higher in Mongolia (14.0%) than in Nepal (2.2%; P < 0.001) in these data sets. In Mongolia (RetCam images), the area under the receiver operating characteristic curve for examination-level plus disease detection was 0.968, and the area under the precision-recall curve was 0.823. In Nepal (Forus images), these values were 0.999 and 0.993, respectively. The ROP VSS was associated with ICROP classification in both datasets (P < 0.001). At the population level, the median VSS was found to be higher in Mongolia (2.7; interquartile range [IQR], 1.3-5.4]) as compared with Nepal (1.9; IQR, 1.2-3.4; P < 0.001). Conclusions: These data provide preliminary evidence of the effectiveness of the i-ROP DL algorithm for ROP screening in neonatal populations in Nepal and Mongolia using multiple camera systems and are useful for consideration in future clinical implementation of artificial intelligence-based ROP screening in low- and middle-income countries.

Synthetic Medical Images for Robust, Privacy-Preserving Training of Artificial Intelligence: Application to Retinopathy of Prematurity Diagnosis.

Purpose: Developing robust artificial intelligence (AI) models for medical image analysis requires large quantities of diverse, well-chosen data that can prove challenging to collect because of privacy concerns, disease rarity, or diagnostic label quality. Collecting image-based datasets for retinopathy of prematurity (ROP), a potentially blinding disease, suffers from these challenges. Progressively growing generative adversarial networks (PGANs) may help, because they can synthesize highly realistic images that may increase both the size and diversity of medical datasets. Design: Diagnostic validation study of convolutional neural networks (CNNs) for plus disease detection, a component of severe ROP, using synthetic data. Participants: Five thousand eight hundred forty-two retinal fundus images (RFIs) collected from 963 preterm infants. Methods: Retinal vessel maps (RVMs) were segmented from RFIs. PGANs were trained to synthesize RVMs with normal, pre-plus, or plus disease vasculature. Convolutional neural networks were trained, using real or synthetic RVMs, to detect plus disease from 2 real RVM test datasets. Main Outcome Measures: Features of real and synthetic RVMs were evaluated using uniform manifold approximation and projection (UMAP). Similarities were evaluated at the dataset and feature level using Fréchet inception distance and Euclidean distance, respectively. CNN performance was assessed via area under the receiver operating characteristic curve (AUC); AUCs were compared via bootstrapping and Delong's test for correlated receiver operating characteristic curves. Confusion matrices were compared using McNemar's chi-square test and Cohen's κ value. Results: The CNN trained on synthetic RVMs showed a significantly higher AUC (0.971; P = 0.006 and P = 0.004) and classified plus disease more similarly to a set of 8 international experts (κ = 0.922) than the CNN trained on real RVMs (AUC = 0.934; κ = 0.701). Real and synthetic RVMs overlapped, by plus disease diagnosis, on the UMAP manifold, showing that synthetic images spanned the disease severity spectrum. Fréchet inception distance and Euclidean distances suggested that real and synthetic RVMs were more dissimilar to one another than real RVMs were to one another, further suggesting that synthetic RVMs were distinct from the training data with respect to privacy considerations. Conclusions: Synthetic datasets may be useful for training robust medical AI models. Furthermore, PGANs may be able to synthesize realistic data for use without protected health information concerns.

Improved Training Efficiency for Retinopathy of Prematurity Deep Learning Models Using Comparison versus Class Labels.

Purpose: To compare the efficacy and efficiency of training neural networks for medical image classification using comparison labels indicating relative disease severity versus diagnostic class labels from a retinopathy of prematurity (ROP) image dataset. Design: Evaluation of diagnostic test or technology. Participants: Deep learning neural networks trained on expert-labeled wide-angle retinal images obtained from patients undergoing diagnostic ROP examinations obtained as part of the Imaging and Informatics in ROP (i-ROP) cohort study. Methods: Neural networks were trained with either class or comparison labels indicating plus disease severity in ROP retinal fundus images from 2 datasets. After training and validation, all networks underwent evaluation using a separate test dataset in 1 of 2 binary classification tasks: normal versus abnormal or plus versus nonplus. Main Outcome Measures: Area under the receiver operating characteristic curve (AUC) values were measured to assess network performance. Results: Given the same number of labels, neural networks learned more efficiently by comparison, generating significantly higher AUCs in both classification tasks across both datasets. Similarly, given the same number of images, comparison learning developed networks with significantly higher AUCs across both classification tasks in 1 of 2 datasets. The difference in efficiency and accuracy between models trained on either label type decreased as the size of the training set increased. Conclusions: Comparison labels individually are more informative and more abundant per sample than class labels. These findings indicate a potential means of overcoming the common obstacle of data variability and scarcity when training neural networks for medical image classification tasks.