AI-Human Hybrid Workflow Enhances Teleophthalmology for the Detection of Diabetic Retinopathy.

Objective: Detection of diabetic retinopathy (DR) outside of specialized eye care settings is an important means of access to vision-preserving health maintenance. Remote interpretation of fundus photographs acquired in a primary care or other nonophthalmic setting in a store-and-forward manner is a predominant paradigm of teleophthalmology screening programs. Artificial intelligence (AI)-based image interpretation offers an alternative means of DR detection. IDx-DR (Digital Diagnostics Inc) is a Food and Drug Administration-authorized autonomous testing device for DR. We evaluated the diagnostic performance of IDx-DR compared with human-based teleophthalmology over 2 and a half years. Additionally, we evaluated an AI-human hybrid workflow that combines AI-system evaluation with human expert-based assessment for referable cases. Design: Prospective cohort study and retrospective analysis. Participants: Diabetic patients ≥ 18 years old without a prior DR diagnosis or DR examination in the past year presenting for routine DR screening in a primary care clinic. Methods: Macula-centered and optic nerve-centered fundus photographs were evaluated by an AI algorithm followed by consensus-based overreading by retina specialists at the Stanford Ophthalmic Reading Center. Detection of more-than-mild diabetic retinopathy (MTMDR) was compared with in-person examination by a retina specialist. Main Outcome Measures: Sensitivity, specificity, accuracy, positive predictive value, and gradability achieved by the AI algorithm and retina specialists. Results: The AI algorithm had higher sensitivity (95.5% sensitivity; 95% confidence interval [CI], 86.7%-100%) but lower specificity (60.3% specificity; 95% CI, 47.7%-72.9%) for detection of MTMDR compared with remote image interpretation by retina specialists (69.5% sensitivity; 95% CI, 50.7%-88.3%; 96.9% specificity; 95% CI, 93.5%-100%). Gradability of encounters was also lower for the AI algorithm (62.5%) compared with retina specialists (93.1%). A 2-step AI-human hybrid workflow in which the AI algorithm initially rendered an assessment followed by overread by a retina specialist of MTMDR-positive encounters resulted in a sensitivity of 95.5% (95% CI, 86.7%-100%) and a specificity of 98.2% (95% CI, 94.6%-100%). Similarly, a 2-step overread by retina specialists of AI-ungradable encounters improved gradability from 63.5% to 95.6% of encounters. Conclusions: Implementation of an AI-human hybrid teleophthalmology workflow may both decrease reliance on human specialist effort and improve diagnostic accuracy. Financial Disclosures: Proprietary or commercial disclosure may be found after the references.

Primary Care 2.0: A Prospective Evaluation of a Novel Model of Advanced Team Care With Expanded Medical Assistant Support.

PURPOSE: Assess effectiveness of Primary Care 2.0: a team-based model that incorporates increased medical assistant (MA) to primary care physician (PCP) ratio, integration of advanced practice clinicians, expanded MA roles, and extended the interprofessional team. METHODS: Prospective, quasi-experimental evaluation of staff/clinician team development and wellness survey data, comparing Primary Care 2.0 to conventional clinics within our academic health care system. We surveyed before the model launch and every 6-9 months up to 24 months post implementation. Secondary outcomes (cost, quality metrics, patient satisfaction) were assessed via routinely collected operational data. RESULTS: Team development significantly increased in the Primary Care 2.0 clinic, sustained across all 3 post implementation time points (+12.2, +8.5, + 10.1 respectively, vs baseline, on the 100-point Team Development Measure) relative to the comparison clinics. Among wellness domains, only "control of work" approached significant gains (+0.5 on a 5-point Likert scale, P = .05), but was not sustained. Burnout did not have statistically significant relative changes; the Primary Care 2.0 site showed a temporal trend of improvement at 9 and 15 months. Reversal of this trend at 2 years corresponded to contextual changes, specifically, reduced MA to PCP staffing ratio. Adjusted models confirmed an inverse relationship between team development and burnout (P <.0001). Secondary outcomes generally remained stable between intervention and comparison clinics with suggestion of labor cost savings. CONCLUSIONS: The Primary Care 2.0 model of enhanced team-based primary care demonstrates team development is a plausible key to protect against burnout, but is not sufficient alone. The results reinforce that transformation to team-based care cannot be a 1-time effort and institutional commitment is integral.