Cross-camera Performance of Deep Learning Algorithms to Diagnose Common Ophthalmic Diseases: A Comparative Study Highlighting Feasibility to Portable Fundus Camera Use.

PURPOSE: To compare the inter-camera performance and consistency of various deep learning (DL) diagnostic algorithms applied to fundus images taken from desktop Topcon and portable Optain cameras. METHODS: Participants over 18 years of age were enrolled between November 2021 and April 2022. Pair-wise fundus photographs from each patient were collected in a single visit; once by Topcon (used as the reference camera) and once by a portable Optain camera (the new target camera). These were analyzed by three previously validated DL models for the detection of diabetic retinopathy (DR), age-related macular degeneration (AMD), and glaucomatous optic neuropathy (GON). Ophthalmologists manually analyzed all fundus photos for the presence of DR and these were referred to as the ground truth. Sensitivity, specificity, the area under the curve (AUC) and agreement between cameras (estimated by Cohen's weighted kappa, K) were the primary outcomes of this study. RESULTS: A total of 504 patients were recruited. After excluding 12 photographs with matching errors and 59 photographs with low quality, 906 pairs of Topcon-Optain fundus photos were available for algorithm assessment. Topcon and Optain cameras had excellent consistency (Κ=0.80) when applied to the referable DR algorithm, while AMD had moderate consistency (Κ=0.41) and GON had poor consistency (Κ=0.32). For the DR model, Topcon and Optain achieved a sensitivity of 97.70% and 97.67% and a specificity of 97.92% and 97.93%, respectively. There was no significant difference between the two camera models (McNemar's test: x2=0.08, p = .78). CONCLUSION: Topcon and Optain cameras had excellent consistency for detecting referable DR, albeit performances for detecting AMD and GON models were unsatisfactory. This study highlights the methods of using pair-wise images to evaluate DL models between reference and new fundus cameras.

Comparing Common Retinal Vessel Caliber Measurement Software with an Automatic Deep Learning System.

PURPOSE: To compare the Retina-based Microvascular Health Assessment System (RMHAS) with Integrative Vessel Analysis (IVAN) for retinal vessel caliber measurement. METHODS: Eligible fundus photographs from the Lingtou Eye Cohort Study were obtained alongside their corresponding participant data. Vascular diameter was automatically measured using IVAN and RMHAS software, and intersoftware variations were assessed by intra-class correlation coefficients (ICC), and 95% confidence intervals (CIs). Scatterplots and Bland-Altman plots assessed the agreement between programs, and a Pearson's correlation test assessed the strength of associations between systemic variables and retinal calibers. An algorithm was proposed to convert measurements between software for interchangeability. RESULTS: ICCs between IVAN and RMHAS were moderate for CRAE and AVR (ICC; 95%CI)(0.62; 0.60 to 0.63 and 0.42; 0.40 to 0.44 respectively) and excellent for CRVE (0.76; 0.75 to 0.77). When comparing retinal vascular calibre measurements between tools, mean differences (MD, 95% confidence intervals) in CRAE, CRVE, and AVR were 22.34 (-7.29 to 51.97 µm),-7.01 (-37.68 to 23.67 µm), and 0.12 (-0.02 to 0.26 µm), respectively. The correlation of systemic parameters with CRAE/CRVE was poor and the correlation of CRAE with age, sex, systolic blood pressure, and CRVE with age, sex, and serum glucose were significantly different between IVAN and RMHAS (p < 0.05). CONCLUSIONS: CRAE and AVR correlated moderately between retinal measurement software systems while CRVE correlated well. Further studies confirming this agreeability and interchangeability in large-scale datasets are needed before softwares are deemed comparable in clinical practice.