Automated Machine Learning for Predicting Diabetic Retinopathy Progression From Ultra-Widefield Retinal Images.

Importance: Machine learning (ML) algorithms have the potential to identify eyes with early diabetic retinopathy (DR) at increased risk for disease progression. Objective: To create and validate automated ML models (autoML) for DR progression from ultra-widefield (UWF) retinal images. Design, Setting and Participants: Deidentified UWF images with mild or moderate nonproliferative DR (NPDR) with 3 years of longitudinal follow-up retinal imaging or evidence of progression within 3 years were used to develop automated ML models for predicting DR progression in UWF images. All images were collected from a tertiary diabetes-specific medical center retinal image dataset. Data were collected from July to September 2022. Exposure: Automated ML models were generated from baseline on-axis 200° UWF retinal images. Baseline retinal images were labeled for progression based on centralized reading center evaluation of baseline and follow-up images according to the clinical Early Treatment Diabetic Retinopathy Study severity scale. Images for model development were split 8-1-1 for training, optimization, and testing to detect 1 or more steps of DR progression. Validation was performed using a 328-image set from the same patient population not used in model development. Main Outcomes and Measures: Area under the precision-recall curve (AUPRC), sensitivity, specificity, and accuracy. Results: A total of 1179 deidentified UWF images with mild (380 [32.2%]) or moderate (799 [67.8%]) NPDR were included. DR progression was present in half of the training set (590 of 1179 [50.0%]). The model's AUPRC was 0.717 for baseline mild NPDR and 0.863 for moderate NPDR. On the validation set for eyes with mild NPDR, sensitivity was 0.72 (95% CI, 0.57-0.83), specificity was 0.63 (95% CI, 0.57-0.69), prevalence was 0.15 (95% CI, 0.12-0.20), and accuracy was 64.3%; for eyes with moderate NPDR, sensitivity was 0.80 (95% CI, 0.70-0.87), specificity was 0.72 (95% CI, 0.66-0.76), prevalence was 0.22 (95% CI, 0.19-0.27), and accuracy was 73.8%. In the validation set, 6 of 9 eyes (75%) with mild NPDR and 35 of 41 eyes (85%) with moderate NPDR progressed 2 steps or more were identified. All 4 eyes with mild NPDR that progressed within 6 months and 1 year were identified, and 8 of 9 (89%) and 17 of 20 (85%) with moderate NPDR that progressed within 6 months and 1 year, respectively, were identified. Conclusions and Relevance: This study demonstrates the accuracy and feasibility of automated ML models for identifying DR progression developed using UWF images, especially for prediction of 2-step or greater DR progression within 1 year. Potentially, the use of ML algorithms may refine the risk of disease progression and identify those at highest short-term risk, thus reducing costs and improving vision-related outcomes.

INTEGRATING MACULAR OPTICAL COHERENCE TOMOGRAPHY WITH ULTRAWIDE-FIELD IMAGING IN A DIABETIC RETINOPATHY TELEMEDICINE PROGRAM USING A SINGLE DEVICE.

PURPOSE: To determine the effect of combined macular spectral-domain optical coherence tomography (SD-OCT) and ultrawide field retinal imaging (UWFI) within a telemedicine program. METHODS: Comparative cohort study of consecutive patients with both UWFI and SD-OCT. Ultrawide field retinal imaging and SD-OOCT were independently evaluated for diabetic macular edema (DME) and nondiabetic macular abnormality. Sensitivity and specificity were calculated with SD-OCT as the gold standard. RESULTS: Four hundred twenty-two eyes from 211 diabetic patients were evaluated. Diabetic macular edema severity by UWFI was as follows: no DME 93.4%, noncenter involved DME (nonciDME) 5.1%, ciDME 0.7%, ungradable DME 0.7%. SD-OCT was ungradable in 0.5%. Macular abnormality was identified in 34 (8.1%) eyes by UWFI and in 44 (10.4%) eyes by SD-OCT. Diabetic macular edema represented only 38.6% of referable macular abnormality identified by SD-OCT imaging. Sensitivity/specificity of UWFI compared with SD-OCT was 59%/96% for DME and 33%/99% for ciDME. Sensitivity/specificity of UWFI compared with SDOCT was 3%/98% for epiretinal membrane. CONCLUSION: Addition of SD-OCT increased the identification of macular abnormality by 29.4%. More than 58.3% of the eyes believed to have any DME on UWF imaging alone were false-positives by SD-OCT. The integration of SD-OCT with UWFI markedly increased detection and reduced false-positive assessments of DME and macular abnormality in a teleophthalmology program.

One-field, two-field and five-field handheld retinal imaging compared with standard seven-field Early Treatment Diabetic Retinopathy Study photography for diabetic retinopathy screening.

BACKGROUND/AIMS: To determine agreement of one-field (1F, macula-centred), two-field (2F, disc-macula) and five-field (5F, macula, disc, superior, inferior and nasal) mydriatic handheld retinal imaging protocols for the assessment of diabetic retinopathy (DR) as compared with standard seven-field Early Treatment Diabetic Retinopathy Study (ETDRS) photography. METHODS: Prospective, comparative instrument validation study. Mydriatic retinal images were taken using three handheld retinal cameras: Aurora (AU; 50° field of view (FOV), 5F), Smartscope (SS; 40° FOV, 5F), and RetinaVue (RV; 60° FOV, 2F) followed by ETDRS photography. Images were evaluated at a centralised reading centre using the international DR classification. Each field protocol (1F, 2F and 5F) was graded independently by masked graders. Weighted kappa (Kw) statistics assessed agreement for DR. Sensitivity (SN) and specificity (SP) for referable diabetic retinopathy (refDR; moderate non-proliferative diabetic retinopathy (NPDR) or worse, or ungradable images) were calculated. RESULTS: Images from 225 eyes of 116 patients with diabetes were evaluated. Severity by ETDRS photography: no DR, 33.3%; mild NPDR, 20.4%; moderate, 14.2%; severe, 11.6%; proliferative, 20.4%. Ungradable rate for DR: ETDRS, 0%; AU: 1F 2.23%, 2F 1.79%, 5F 0%; SS: 1F 7.6%, 2F 4.0%, 5F 3.6%; RV: 1F 6.7%, 2F 5.8%. Agreement rates of DR grading between handheld retinal imaging and ETDRS photography were (Kw, SN/SP refDR) AU: 1F 0.54, 0.72/0.92; 2F 0.59, 0.74/0.92; 5F 0.75, 0.86/0.97; SS: 1F 0.51, 0.72/0.92; 2F 0.60, 0.75/0.92; 5F 0.73, 0.88/0.92; RV: 1F 0.77, 0.91/0.95; 2F 0.75, 0.87/0.95. CONCLUSION: When using handheld devices, the addition of peripheral fields decreased the ungradable rate and increased SN and SP for refDR. These data suggest the benefit of additional peripheral fields in DR screening programmes that use handheld retinal imaging.

Comparisons of Handheld Retinal Imaging with Optical Coherence Tomography for the Identification of Macular Pathology in Patients with Diabetes.

INTRODUCTION: Handheld retinal imaging cameras are relatively inexpensive and highly portable devices that have the potential to significantly expand diabetic retinopathy (DR) screening, allowing a much broader population to be evaluated. However, it is essential to evaluate if these devices can accurately identify vision-threatening macular diseases if DR screening programs will rely on these instruments. Thus, the purpose of this study was to evaluate the detection of diabetic macular pathology using monoscopic macula-centered images using mydriatic handheld retinal imaging compared with spectral domain optical coherence tomography (SDOCT). METHODS: Mydriatic 40°-60° macula-centered images taken with 3 handheld retinal imaging devices (Aurora [AU], SmartScope [SS], RetinaVue 700 [RV]) were compared with the Cirrus 6000 SDOCT taken during the same visit. Images were evaluated for the presence of diabetic macular edema (DME) on monoscopic fundus photographs adapted from Early Treatment Diabetic Retinopathy Study (ETDRS) definitions (no DME, noncenter-involved DME [non-ciDME], and center-involved DME [ciDME]). Sensitivity, specificity, positive predictive value, and negative predictive value were calculated for each device with SDOCT as gold standard. RESULTS: Severity by ETDRS photos: no DR 33.3%, mild NPDR 20.4%, moderate 14.2%, severe 11.6%, proliferative 20.4%, and ungradable for DR 0%; no DME 83.1%, non-ciDME 4.9%, ciDME 12.0%, and ungradable for DME 0%. Gradable images by SDOCT (N = 217, 96.4%) showed no DME in 75.6%, non-ciDME in 9.8%, and ciDME in 11.1%. The ungradable rate for images (poor visualization in >50% of the macula) was AU: 0.9%, SS: 4.4%, and RV: 6.2%. For DME, sensitivity and specificity were similar across devices (0.5-0.64, 0.93-0.97). For nondiabetic macular pathology (ERM, pigment epithelial detachment, traction retinal detachment) across all devices, sensitivity was low to moderate (0.2-0.5) but highly specific (0.93-1.00). CONCLUSIONS: Compared to SDOCT, handheld macular imaging attained high specificity but low sensitivity in identifying macular pathology. This suggests the importance of SDOCT evaluation for patients suspected to have DME on fundus photography, leading to more appropriate referral refinement.

Performance of Automated Machine Learning for Diabetic Retinopathy Image Classification from Multi-field Handheld Retinal Images.

PURPOSE: To create and validate code-free automated deep learning models (AutoML) for diabetic retinopathy (DR) classification from handheld retinal images. DESIGN: Prospective development and validation of AutoML models for DR image classification. PARTICIPANTS: A total of 17 829 deidentified retinal images from 3566 eyes with diabetes, acquired using handheld retinal cameras in a community-based DR screening program. METHODS: AutoML models were generated based on previously acquired 5-field (macula-centered, disc-centered, superior, inferior, and temporal macula) handheld retinal images. Each individual image was labeled using the International DR and diabetic macular edema (DME) Classification Scale by 4 certified graders at a centralized reading center under oversight by a senior retina specialist. Images for model development were split 8-1-1 for training, optimization, and testing to detect referable DR ([refDR], defined as moderate nonproliferative DR or worse or any level of DME). Internal validation was performed using a published image set from the same patient population (N = 450 images from 225 eyes). External validation was performed using a publicly available retinal imaging data set from the Asia Pacific Tele-Ophthalmology Society (N = 3662 images). MAIN OUTCOME MEASURES: Area under the precision-recall curve (AUPRC), sensitivity (SN), specificity (SP), positive predictive value (PPV), negative predictive value (NPV), accuracy, and F1 scores. RESULTS: Referable DR was present in 17.3%, 39.1%, and 48.0% of the training set, internal validation, and external validation sets, respectively. The model's AUPRC was 0.995 with a precision and recall of 97% using a score threshold of 0.5. Internal validation showed that SN, SP, PPV, NPV, accuracy, and F1 scores were 0.96 (95% confidence interval [CI], 0.884-0.99), 0.98 (95% CI, 0.937-0.995), 0.96 (95% CI, 0.884-0.99), 0.98 (95% CI, 0.937-0.995), 0.97, and 0.96, respectively. External validation showed that SN, SP, PPV, NPV, accuracy, and F1 scores were 0.94 (95% CI, 0.929-0.951), 0.97 (95% CI, 0.957-0.974), 0.96 (95% CI, 0.952-0.971), 0.95 (95% CI, 0.935-0.956), 0.97, and 0.96, respectively. CONCLUSIONS: This study demonstrates the accuracy and feasibility of code-free AutoML models for identifying refDR developed using handheld retinal imaging in a community-based screening program. Potentially, the use of AutoML may increase access to machine learning models that may be adapted for specific programs that are guided by the clinical need to rapidly address disparities in health care delivery. FINANCIAL DISCLOSURE(S): Proprietary or commercial disclosure may be found after the references.

Effect of Accessible Nonmydriatic Retinal Imaging on Diabetic Retinopathy Surveillance Rates.

Purpose: To evaluate the impact on surveillance rates for diabetic retinopathy (DR) by providing nonmydriatic retinal imaging as part of comprehensive diabetes care at no cost to patients or insurers. Methods: A retrospective comparative cohort study was designed. Patients were imaged from April 1, 2016 to March 31, 2017 at a tertiary diabetes-specific academic medical center. Retinal imaging was provided without additional cost beginning October 16, 2016. Images were evaluated for DR and diabetic macular edema using standard protocol at a centralized reading center. Diabetes surveillance rates before and after no-cost imaging were compared. Results: A total of 759 and 2,080 patients respectively were imaged before and after offering no-cost retinal imaging. The difference represents a 274% increase in the number of patients screened. Furthermore, there was a 292% and 261% increase in the number of eyes with mild DR and referable DR, respectively. In the comparative 6-month period, 92 additional cases of proliferative DR were identified, estimated to prevent 6.7 cases of severe visual loss with annual cost savings of $180,230 (estimated yearly cost of severe vision loss per person: $26,900). In patients with referable DR, self-awareness was low, with no significant difference in the before and after groups (39.4% vs. 43.8%, p = 0.3725). Conclusions: Providing retinal imaging as part of comprehensive diabetes care substantially increased the number of patients identified by nearly threefold. The data suggest that the removal of out-of-pocket costs substantially increased patient surveillance rates, which may translate to improved long-term patient outcomes.

Comparisons of handheld retinal imaging devices with ultrawide field images for determining diabetic retinopathy severity.

PURPOSE: To compare diabetic retinopathy (DR) severity identified on handheld retinal imaging with ultrawide field (UWF) images. METHODS: Mydriatic images of 225 eyes of 118 diabetic patients were prospectively imaged with the Aurora (AU) handheld retinal camera [5-field protocol (macula-centred, disc-centred, temporal, superior, inferior)] and compared with UWF images. Images were classified based on the international classification for DR. Sensitivity, specificity, kappa statistics (K/Kw) were calculated on an eye and person-level. RESULTS: Distribution of DR severity by AU/UWF images (%) by eye was no DR 41.3/36.0, mild non-proliferative DR (NPDR) 18.7/17.8, moderate 10.2/10.7, severe 16.4/15.1, proliferative DR (PDR) 13.3/20.4. Agreement between UWF and AU was exact in 64.4%, within 1-step 90.7%, k = 0.55 (95% CI:0.45-0.65), and kw = 0.79 (95% CI:0.73-0.85) by eye, and exact in 68%, within 1-step 92.9%, k = 0.58 (95% CI:0.50-0.66), and kw = 0.76 (95% CI:0.70-0.81) by person. Sensitivity/specificity for any DR, refDR, vtDR and PDR were as follows: 0.90/0.83, 0.90/0.97, 0.82/0.95 and 0.69/1.00 by person and 0.86/0.90, 0.84/0.98, 0.75/0.95 and 0.63/0.99 by eye. Handheld imaging missed 37% (17/46) eyes and 30.8% (8/26) persons with PDR. Only 3.9% (1/26) persons or 6.5% (3/46) eyes with PDR were missed if a referral threshold of moderate NPDR was used. CONCLUSIONS: Data from this study show that comparing UWF and handheld images, when PDR was the referral threshold for handheld devices, 37.0% of eyes or 30.8% of patients with PDR were missed. Due to the identification of neovascular lesions outside of the handheld fields, lower referral thresholds are needed if handheld devices are used.

Circulatory Biomarkers and Diabetic Retinopathy in Racial and Ethnic Populations.

Clinical staging systems for diagnosis and treatment of diabetic retinopathy (DR) must closely relate to endpoints that are both relevant for patients and feasible for physicians to implement. Current DR staging systems for clinical eye care and research provide detailed phenotypic characterization to predict patient outcomes in diabetes but have limitations. Biochemical biomarkers provide a rich pool of potential candidates for new DR staging systems that can be readily measured in accessible fluids. Circulating biomarkers that are specific to the retina and relate to angiogenesis and inflammation have been suggested as relevant for DR. Although there is a lack of multi-ethnic studies evaluating circulatory biomarkers in DR, variability in circulatory biomarkers have been reported in people from different ethnic and racial backgrounds. Therefore, there is a need for future studies to evaluate individual or combinations of biomarkers in diverse populations with DR from different ethnic and racial backgrounds.

Bias and Non-Diversity of Big Data in Artificial Intelligence: Focus on Retinal Diseases.

Artificial intelligence (AI) applications in healthcare will have a potentially far-reaching impact on patient care, however issues regarding algorithmic bias and fairness have recently surfaced. There is a recognized lack of diversity in the available ophthalmic datasets, with 45% of the global population having no readily accessible representative images, leading to potential misrepresentations of their unique anatomic features and ocular pathology. AI applications in retinal disease may show less accuracy with underrepresented populations that may further widen the gap of health inequality if left unaddressed. Beyond disease symptomatology, social determinants of health must be integrated into our current paradigms of disease understanding, with the goal of more personalized care. AI has the potential to decrease global healthcare inequality, but it will need to be based on a more diverse, transparent and responsible use of healthcare data.

Association of Maximizing Visible Retinal Area by Manual Eyelid Lifting With Grading of Diabetic Retinopathy Severity and Detection of Predominantly Peripheral Lesions When Using Ultra-Widefield Imaging.

IMPORTANCE: Methods that increase visible retinal area (VRA; measured in millimeters squared) may improve identification of diabetic retinopathy (DR) lesions. OBJECTIVE: To evaluate the association of dilation and manual eyelid lifting (MLL) with VRA on ultra-widefield imaging (UWFI) and the association of VRA with grading of DR severity and detection of predominantly peripheral lesions (PPLs). DESIGN, SETTING, AND PARTICIPANTS: Retrospective, comparative case-control study at the Joslin Diabetes Center, Boston, Massachusetts. Nonmydriatic UWFI with MLL was acquired from a DR teleophthalmology program (Joslin Vision Network [JVN]). A second cohort of mydriatic UWFI was acquired at an academic retina practice (Beetham Eye Institute [BEI]) from November 6, 2017, to November 6, 2018, and with MLL thereafter until November 6, 2019. Fully automated algorithms determined VRA and hemorrhage and/or microaneurysm (HMA) counts. Predominantly peripheral lesions and HMAs were defined as present when at least 1 field had greater HMA number in the peripheral retina than within the corresponding Early Treatment Diabetic Retinopathy Study field. Participants included 3014 consecutive patients (5919 eyes) undergoing retinal imaging at JVN and BEI. EXPOSURES: Dilation and MLL performed at the time of UWFI. MAIN OUTCOMES AND MEASURES: Visible retinal area, DR severity, and presence of PPLs. RESULTS: Of the 3014 participants, mean (SD) age was 56.1 (14.5) years, 1302 (43.2%) were female, 2450 (81.3%) were White, and mean (SD) diabetes duration was 15.9 (11.4) years. All images from 5919 eyes with UWFI were analyzed. Mean (SD) VRA was 665.1 (167.6) mm2 for all eyes (theoretical maximal VRA, 923.9 mm2), 550.8 (240.7) mm2 for nonmydriatic JVN with MLL (1418 eyes [24.0%]), 688.1 (119.9) mm2 for mydriatic BEI images (3650 eyes [61.7%]), and 757.0 (69.7) mm2 for mydriatic and MLL BEI images (851 eyes [14.4%]). Dilation increased VRA by 25% (P < .001) and MLL increased VRA an additional 10% (P < .001). Nonmydriatic MLL increased VRA by 11.0%. With MLL, HMA counts in UWFI fields increased by 41.7% (from 4.8 to 6.8; P < .001). Visible retinal area was moderately associated with increasing PPL-HMA overall and in each cohort (all, r = 0.33; BEI, r = 0.29; JVN, r = 0.36; P < .001). In JVN images, increasing VRA was associated with more PPL-HMA (quartile 1 [Q1], 23.7%; Q2, 45.8%; Q3, 60.6%; and Q4, 69.2%; P < .001). CONCLUSIONS AND RELEVANCE: Using fully automated VRA and HMA detection algorithms, pupillary dilation and eyelid lifting were shown to substantially increase VRA and PLL-HMA detection. Given the importance of HMA and PPL for determining risk of DR progression, these findings emphasize the importance of maximizing VRA for optimal risk assessment in clinical trials and teleophthalmology programs.

Biomarkers for Progression in Diabetic Retinopathy: Expanding Personalized Medicine through Integration of AI with Electronic Health Records.

The goal of personalized diabetes eye care is to accurately predict in real-time the risk of diabetic retinopathy (DR) progression and visual loss. The use of electronic health records (EHR) provides a platform for artificial intelligence (AI) algorithms that predict DR progression to be incorporated into clinical decision-making. By implementing an algorithm on data points from each patient, their risk for retinopathy progression and visual loss can be modeled, allowing them to receive timely treatment. Data can guide algorithms to create models for disease and treatment that may pave the way for more personalized care. Currently, there exist numerous challenges that need to be addressed before reliably building and deploying AI algorithms, including issues with data quality, privacy, intellectual property, and informed consent.