Validity of Administrative Claims and Electronic Health Registry Data From a Single Practice for Eye Health Surveillance.

Importance: Diagnostic information from administrative claims and electronic health record (EHR) data may serve as an important resource for surveillance of vision and eye health, but the accuracy and validity of these sources are unknown. Objective: To estimate the accuracy of diagnosis codes in administrative claims and EHRs compared to retrospective medical record review. Design, Setting, and Participants: This cross-sectional study compared the presence and prevalence of eye disorders based on diagnostic codes in EHR and claims records vs clinical medical record review at University of Washington-affiliated ophthalmology or optometry clinics from May 2018 to April 2020. Patients 16 years and older with an eye examination in the previous 2 years were included, oversampled for diagnosed major eye diseases and visual acuity loss. Exposures: Patients were assigned to vision and eye health condition categories based on diagnosis codes present in their billing claims history and EHR using the diagnostic case definitions of the US Centers for Disease Control and Prevention Vision and Eye Health Surveillance System (VEHSS) as well as clinical assessment based on retrospective medical record review. Main Outcome and Measures: Accuracy was measured as area under the receiver operating characteristic curve (AUC) of claims and EHR-based diagnostic coding vs retrospective review of clinical assessments and treatment plans. Results: Among 669 participants (mean [range] age, 66.1 [16-99] years; 357 [53.4%] female), identification of diseases in billing claims and EHR data using VEHSS case definitions was accurate for diabetic retinopathy (claims AUC, 0.94; 95% CI, 0.91-0.98; EHR AUC, 0.97; 95% CI, 0.95-0.99), glaucoma (claims AUC, 0.90; 95% CI, 0.88-0.93; EHR AUC, 0.93; 95% CI, 0.90-0.95), age-related macular degeneration (claims AUC, 0.87; 95% CI, 0.83-0.92; EHR AUC, 0.96; 95% CI, 0.94-0.98), and cataracts (claims AUC, 0.82; 95% CI, 0.79-0.86; EHR AUC, 0.91; 95% CI, 0.89-0.93). However, several condition categories showed low validity with AUCs below 0.7, including diagnosed disorders of refraction and accommodation (claims AUC, 0.54; 95% CI, 0.49-0.60; EHR AUC, 0.61; 95% CI, 0.56-0.67), diagnosed blindness and low vision (claims AUC, 0.56; 95% CI, 0.53-0.58; EHR AUC, 0.57; 95% CI, 0.54-0.59), and orbital and external diseases (claims AUC, 0.63; 95% CI, 0.57-0.69; EHR AUC, 0.65; 95% CI, 0.59-0.70). Conclusion and Relevance: In this cross-sectional study of current and recent ophthalmology patients with high rates of eye disorders and vision loss, identification of major vision-threatening eye disorders based on diagnosis codes in claims and EHR records was accurate. However, vision loss, refractive error, and other broadly defined or lower-risk disorder categories were less accurately identified by diagnosis codes in claims and EHR data.

Comparing Telephone Survey Responses to Best-Corrected Visual Acuity to Estimate the Accuracy of Identifying Vision Loss: Validation Study.

BACKGROUND: Self-reported questions on blindness and vision problems are collected in many national surveys. Recently released surveillance estimates on the prevalence of vision loss used self-reported data to predict variation in the prevalence of objectively measured acuity loss among population groups for whom examination data are not available. However, the validity of self-reported measures to predict prevalence and disparities in visual acuity has not been established. OBJECTIVE: This study aimed to estimate the diagnostic accuracy of self-reported vision loss measures compared to best-corrected visual acuity (BCVA), inform the design and selection of questions for future data collection, and identify the concordance between self-reported vision and measured acuity at the population level to support ongoing surveillance efforts. METHODS: We calculated accuracy and correlation between self-reported visual function versus BCVA at the individual and population level among patients from the University of Washington ophthalmology or optometry clinics with a prior eye examination, randomly oversampled for visual acuity loss or diagnosed eye diseases. Self-reported visual function was collected via telephone survey. BCVA was determined based on retrospective chart review. Diagnostic accuracy of questions at the person level was measured based on the area under the receiver operator curve (AUC), whereas population-level accuracy was determined based on correlation. RESULTS: The survey question, "Are you blind or do you have serious difficulty seeing, even when wearing glasses?" had the highest accuracy for identifying patients with blindness (BCVA ≤20/200; AUC=0.797). The highest accuracy for detecting any vision loss (BCVA <20/40) was achieved by responses of "fair," "poor," or "very poor" to the question, "At the present time, would you say your eyesight, with glasses or contact lenses if you wear them, is excellent, good, fair, poor, or very poor" (AUC=0.716). At the population level, the relative relationship between prevalence based on survey questions and BCVA remained stable for most demographic groups, with the only exceptions being groups with small sample sizes, and these differences were generally not significant. CONCLUSIONS: Although survey questions are not considered to be sufficiently accurate to be used as a diagnostic test at the individual level, we did find relatively high levels of accuracy for some questions. At the population level, we found that the relative prevalence of the 2 most accurate survey questions were highly correlated with the prevalence of measured visual acuity loss among nearly all demographic groups. The results of this study suggest that self-reported vision questions fielded in national surveys are likely to yield an accurate and stable signal of vision loss across different population groups, although the actual measure of prevalence from these questions is not directly analogous to that of BCVA.

Flipped classroom approach to global outreach: cross-cultural teaching of horizontal strabismus to Chinese ophthalmology residents.

AIM: To evaluate the flipped classroom model for teaching horizontal strabismus didactics in an ophthalmology residency program in China as part of a visiting professorship from the United States. METHODS: Residents from an ophthalmology residency program in China were invited to participate in flipped classroom sessions taught by an experienced American ophthalmology faculty in 2018. Residents were instructed to watch a pre-class video lecture prior to the in-class-case-based activity. Content tests (5 Ophthalmic Knowledge Assessment Program style questions) and surveys were administered before and after the classroom sessions (100% response rate). These results were compared to that of an American cohort who were taught the same content. RESULTS: The Chinese cohort of 12 residents preferred the flipped classroom to the traditional classroom at higher rates than the American cohort of 40 residents (92% vs 55%, P=0.04) and felt that all ophthalmology topics would be appropriate for the flipped classroom teaching style (P-values between 0.008 and <0.001). In both Chinese and American cohorts, we found that the exotropia curriculum saw a small but significant improvement in performance following the flipped classroom session (P=0.025 for Chinese residents; P=0.001 for US residents), whereas scores in both groups for the esotropia course did not significantly improve. CONCLUSION: This is the first study to evaluate the flipped classroom model implemented by a visiting ophthalmology professor in a global outreach setting. The flipped classroom sessions are viewed favorably by the Chinese residents relative to the US cohort with a modest impact on knowledge. Decreased in-person interpreter requirement and increased student engagement make this model valuable in cross-cultural visiting professorship settings. Finally, the flipped classroom may lend itself well to a virtual format to prevent the transmission of COVID-19, although such a format requires further study.

Predictors of Long-term Ophthalmic Complications after Closed Globe Injuries Using the Intelligent Research in Sight (IRIS®) Registry.

Purpose: To identify clinical factors associated with the need for future surgical intervention following closed globe ocular trauma. Design: Retrospective cohort study. Subjects Participants and/or Controls: Patients in the American Academy of Ophthalmology Intelligent Research in Sight (IRIS®) Registry with a diagnosis of closed globe ocular trauma occurring between 2013 and 2019, identified using International Classification of Disease, 10th Revision and Systematized Nomenclature of Medicine codes. Methods: Diagnosis codes were used to identify multiple concomitant diagnoses present on the date of closed globe ocular trauma. Survival analyses were performed for each outcome of interest, and linear regression was used to identify clinical factors associated with the risk of surgical intervention. Main Outcome Measures: Outcomes included retinal break treatment, retinal detachment (RD) repair, retinal break treatment or RD repair, glaucoma surgery, and cataract surgery. Results: Of the 206 807 patients with closed globe ocular trauma, 9648 underwent surgical intervention during the follow-up period (mean, 444 days): 1697 (0.8%) had RD repair, 1658 (0.8%) had retinal break treatment, 600 (0.3%) had glaucoma surgery, and 5693 (2.8%) had cataract surgery. Traumatic cataract was the strongest risk factor for cataract surgery (hazard ratio, 13.0; 95% confidence interval, 10.8-15.6), traumatic hyphema showed highest risk for glaucoma surgery (7.24; 4.60-11.4), and vitreous hemorrhage was the strongest risk factor for retinal break treatment and detachment repair (11.01; 9.18-13.2 and 14.2; 11.5-17.6, respectively) during the first 60 days after trauma date. Vitreous hemorrhage was a risk factor for cataract surgery at > 60 days after trauma date only. Iris-angle injury was the strongest risk factor for glaucoma surgery > 60 days after trauma, while vitreous hemorrhage remained the strongest factor for retinal break treatment and detachment repair at > 60 days. Traumatic hyphema was a risk factor for all surgical outcomes during all follow-up intervals. Conclusions: Diagnosis of concomitant traumatic cataract, vitreous hemorrhage, traumatic hyphema, and other risk factors may increase the likelihood of requiring surgical intervention after closed globe ocular trauma.

Training Deep Learning Models to Work on Multiple Devices by Cross-Domain Learning with No Additional Annotations.

PURPOSE: To create an unsupervised cross-domain segmentation algorithm for segmenting intraretinal fluid and retinal layers on normal and pathologic macular OCT images from different manufacturers and camera devices. DESIGN: We sought to use generative adversarial networks (GANs) to generalize a segmentation model trained on one OCT device to segment B-scans obtained from a different OCT device manufacturer in a fully unsupervised approach without labeled data from the latter manufacturer. PARTICIPANTS: A total of 732 OCT B-scans from 4 different OCT devices (Heidelberg Spectralis, Topcon 1000, Maestro2, and Zeiss Plex Elite 9000). METHODS: We developed an unsupervised GAN model, GANSeg, to segment 7 retinal layers and intraretinal fluid in Topcon 1000 OCT images (domain B) that had access only to labeled data on Heidelberg Spectralis images (domain A). GANSeg was unsupervised because it had access only to 110 Heidelberg labeled OCTs and 556 raw and unlabeled Topcon 1000 OCTs. To validate GANSeg segmentations, 3 masked graders manually segmented 60 OCTs from an external Topcon 1000 test dataset independently. To test the limits of GANSeg, graders also manually segmented 3 OCTs from Zeiss Plex Elite 9000 and Topcon Maestro2. A U-Net was trained on the same labeled Heidelberg images as baseline. The GANSeg repository with labeled annotations is at https://github.com/uw-biomedical-ml/ganseg. MAIN OUTCOME MEASURES: Dice scores comparing segmentation results from GANSeg and the U-Net model with the manual segmented images. RESULTS: Although GANSeg and U-Net achieved comparable Dice scores performance as human experts on the labeled Heidelberg test dataset, only GANSeg achieved comparable Dice scores with the best performance for the ganglion cell layer plus inner plexiform layer (90%; 95% confidence interval [CI], 68%-96%) and the worst performance for intraretinal fluid (58%; 95% CI, 18%-89%), which was statistically similar to human graders (79%; 95% CI, 43%-94%). GANSeg significantly outperformed the U-Net model. Moreover, GANSeg generalized to both Zeiss and Topcon Maestro2 swept-source OCT domains, which it had never encountered before. CONCLUSIONS: GANSeg enables the transfer of supervised deep learning algorithms across OCT devices without labeled data, thereby greatly expanding the applicability of deep learning algorithms.

Detection of Nonexudative Macular Neovascularization on Structural OCT Images Using Vision Transformers.

Purpose: A deep learning model was developed to detect nonexudative macular neovascularization (neMNV) using OCT B-scans. Design: Retrospective review of a prospective, observational study. Participants: Normal control eyes and patients with age-related macular degeneration (AMD) with and without neMNV. Methods: Swept-source OCT angiography (SS-OCTA) imaging (PLEX Elite 9000, Carl Zeiss Meditec, Inc) was performed using the 6 × 6-mm scan pattern. Individual B-scans were annotated to distinguish between drusen and the double-layer sign (DLS) associated with the neMNV. The machine learning model was tested on a dataset graded by humans, and model performance was compared with the human graders. Main Outcome Measures: Intersection over Union (IoU) score was measured to evaluate segmentation network performance. Area under the receiver operating characteristic curve values, sensitivity, specificity, and positive predictive value (PPV) and negative predictive value (NPV) were measured to assess the performance of the final classification performance. Chance-corrected agreement between the algorithm and the human grader determinations was measured with Cohen's kappa. Results: A total of 251 eyes from 210 patients, including 182 eyes with DLS and 115 eyes with drusen, were used for model training. Of 125 500 B-scans, 6879 B-scans were manually annotated. A vision transformer segmentation model was built to extract DLS and drusen from B-scans. The extracted prediction masks from all B-scans in a volume were projected to an en face image, and an eye-level projection map was obtained for each eye. A binary classification algorithm was established to identify eyes with neMNV from the projection map. The algorithm achieved 82%, 90%, 79%, and 91% sensitivity, specificity, PPV, and NPV, respectively, on a separate test set of 100 eyes that were evaluated by human graders in a previous study. The area under the curve value was calculated as 0.91 (95% confidence interval, 0.85-0.98). The results of the algorithm showed excellent agreement with the senior human grader (kappa = 0.83, P < 0.001) and moderate agreement with the junior grader consensus (kappa = 0.54, P < 0.001). Conclusions: Our network (code is available at https://github.com/uw-biomedical-ml/double_layer_vit) was able to detect the presence of neMNV from structural B-scans alone by applying a purely transformer-based model.

Visual Field Endpoints for Neuroprotective Trials: A Case for AI-Driven Patient Enrichment.

PURPOSE: To evaluate whether an artificial intelligence (AI) model can better select candidates that would demonstrate visual field (VF) progression, in order to shorten the duration or the number of patients needed for a clinical trial. DESIGN: Retrospective cohort study. METHODS: 7428 eyes of 3871 patients from the University of Washington Department of Ophthalmology VF Dataset were included. Progression was defined as at least 5 locations with >7 dB of change compared with baseline on 2 consecutive tests. Progression for all patients, a subgroup of the fastest progressing based on survival curves, and patients selected based on an elastic net Cox regression model were compared. The model was trained on pointwise threshold deviation values of the first VF, age, gender, laterality, and the mean total deviation (MD) at baseline. RESULTS: A total of 13% of all patients met the criteria for progression at 5 years. Differences in survival were observed when stratified by MD and age (P < .0001). Those at risk of progression included patients aged 60 to 80 years with an initial MD < -5.0. This subgroup decreased the sample size required to detect progression compared with the entire cohort. The AI model-selected patients required the lowest number of patients for all effect sizes and trial lengths. For a trial length of 3 years and effect size of 30%, the number of patients required was 1656 (95% CI, 1638-1674), 903 (95% CI, 884-922), and 636 (95% CI, 625-646) for the entire cohort, the subgroup, and the model-selected patients, respectively. CONCLUSION: An AI model can identify high-risk patients to substantially reduce the number of patients needed or study duration required to meet clinical trial endpoints.

UWHVF: A Real-World, Open Source Dataset of Perimetry Tests From the Humphrey Field Analyzer at the University of Washington.

Purpose: This article describes the Humphrey field analyzer (HFA) dataset from the Department of Ophthalmology at the University of Washington. Methods: Pointwise sensitivities were extracted from HFA 24-2, stimulus III visual fields (VF). Total deviation (TD), mean TD (MTD), pattern deviation, and pattern standard deviation (PSD) were calculated. Progression analysis was performed with simple linear regression on global, regional, and pointwise values for VF series with greater than four tests spanning at least four months. VF data were extracted independently of clinical information except for patient age, gender, and laterality. Results: This dataset includes 28,943 VFs from 7248 eyes of 3871 patients. Progression was calculated for 2985 eyes from 1579 patients. Median [interquartile range] age was 64 years [54, 73], and follow-up was 2.49 years [1.11, 5.03]. Baseline MTD was -4.51 dB [-8.01, -2.65], and baseline PSD was 2.41 dB [1.7, 5.34]. Conclusion: MTD was found to decrease by -0.10 dB/yr [-0.40, 0.11] in eyes for which progression analysis was able to be performed. VFs with deep localized defects, PSD > 12 dB and MTD -15 dB to -25 dB, were plotted, visually inspected, and found to be consistent with neurologic or glaucomatous VFs from patients. For a small number of tests, extracted sensitivity values were compared to corresponding printouts and confirmed to match. Translational Relevance: This open access pointwise VF dataset serves as a source of raw data for investigation such as VF behavior, clinical comparisons to trials, and development of new machine learning algorithms.

The flipped-classroom approach to teaching horizontal strabismus in ophthalmology residency: a multicentered randomized controlled study.

BACKGROUND: The flipped-classroom involves watching prerecorded lectures at home followed by group learning exercises within the classroom. This study compares the flipped classroom approach with the traditional classroom for teaching horizontal strabismus didactics in ophthalmology residency. METHODS: In this multicenter, randomized controlled survey study from October 2017 to July 2018, 110 ophthalmology residents were taught esotropia and exotropia sequentially, randomized by order and classroom style. Flipped classroom participants were assigned a preclass video lecture prior to the in-class case-based activity. The traditional classroom included a preparatory reading assignment and an in-person lecture. Residents completed three identical 5-question assessments (pretest, post-test, and 3-month retention) and surveys for each classroom. The primary outcome measured residents' preferences for classroom styles; the secondary outcome compared knowledge acquisition. RESULTS: In our study cohort, the flipped classroom resulted in greater at-home preparation than the traditional classroom (P = 0.001) and was preferred by 33 of 53 residents (62%); 45 of 53 (85%) wished to see the flipped classroom used at least 25% of the time. The exotropia flipped classroom scored higher than traditional classroom on the pretest (3.71/5 [74%] vs 2.87/5 [57%]; P < 0.001) and post-test (4.53/5 [91%] vs 4.13/5 [83%]; P = 0.01) but not the 3-month retention test (3.53/5 [71%] vs 3.37/5 [67%]; P = 0.48). The esotropia classroom styles did not differ on pre- or post-test but demonstrated higher scores for the traditional classroom at 3-month retention (3.43/5 [69%] vs 2.92/5 [58%]; P = 0.03). Advantages cited for flipped classroom include being interactive and engaging while incentivizing better classroom preparation. CONCLUSIONS: The flipped classroom method was received favorably by trainees and may complement traditional methods of teaching.

PeriorbitAI: Artificial Intelligence Automation of Eyelid and Periorbital Measurements.

PURPOSE: To develop a deep learning semantic segmentation network to automate the assessment of 8 periorbital measurements DESIGN: Development and validation of an artificial intelligence (AI) segmentation algorithm METHODS: A total of 418 photographs of periorbital areas were used to train a deep learning semantic segmentation model to segment iris, aperture, and brow areas. These data were used to develop a post-processing algorithm that measured margin reflex distance (MRD) 1 and 2, medial canthal height (MCH), lateral canthal height (LCH), medial brow height (MBH), lateral brow height (LBH), medial intercanthal distance (MID), and lateral intercanthal distance (LID). The algorithm validity was evaluated on a prospective hold-out test set against 3 graders. The main outcome measures were dice coefficient, mean absolute difference, intraclass correlation coefficient, and Bland-Altman analysis. A smartphone video was also segmented and evaluated as proof of concept. RESULTS: The AI algorithm performed in close agreement with all human graders, with a mean absolute difference of 0.5 mm for MRD1, MRD2, LCH, and MCH. The mean absolute difference between graders is approximately 1.5-2 mm for LBH and MBH and approximately 2-4 mm for MID and LID. The 95% confidence intervals for all graders overlapped in most cases, demonstrating that the algorithm performs similarly to human graders. The segmentation of a smartphone video demonstrated that MRD1 can be dynamically measured. CONCLUSIONS: We present, to our knowledge, the first open-sourced, artificial intelligence system capable of automating static and dynamic periorbital measurements. A fully automated tool stands to transform the delivery of clinical care and quantification of surgical outcomes.