ABSTRACT
AIMS: A diabetic eye screening programme has huge value in reducing avoidable sight loss by identifying diabetic retinopathy at a stage when it can be treated. Artificial intelligence automated systems can be used for diabetic eye screening but are not employed in the national English Diabetic Eye Screening Programme. The aim was to report the performance of a commercially available deep-learning artificial intelligence software in a large English population. METHODS: 9817 anonymised image sets from 10,000 consecutive diabetic eye screening episodes were presented to an artificial intelligence software. The sensitivity and specificity of the artificial intelligence system for detecting diabetic retinopathy were determined using the diabetic eye screening programme manual grade according to national protocols as the reference standard. RESULTS: For no diabetic retinopathy versus any diabetic retinopathy, the sensitivity of the artificial intelligence grading system was 69.7% and specificity 92.2%. The performance of the artificial intelligence system was superior for no or mild diabetic retinopathy versus significant or referrable diabetic retinopathy with a sensitivity of 95.4% and specificity of 92.0%. No cases were identified in which the artificial intelligence grade had missed significant diabetic retinopathy. CONCLUSION: The performance of a commercially available deep-learning artificial intelligence system for identifying diabetic retinopathy in an English national Diabetic Eye Screening Programme is presented. Using the pre-defined settings artificial intelligence performance was highest when identifying diabetic retinopathy which requires an action by the screening programme.
Subject(s)
Diabetes Mellitus , Diabetic Retinopathy , Humans , Artificial Intelligence , Mass Screening/methods , Software , Sensitivity and Specificity , Diabetic Retinopathy/diagnosis , Diabetic Retinopathy/epidemiologyABSTRACT
AIM: To assess the accuracy of an artificial intelligence (AI) based software (RetCAD, Thirona, The Netherlands) to identify and grade age-related macular degeneration (AMD) and diabetic retinopathy (DR) simultaneously based on fundus photos. METHODS: This prospective study included 1245 eyes of 630 patients attending an ophthalmology day-care clinic. Fundus photos were acquired and parallel graded by the RetCAD AI software and by an expert reference examiner for image quality, and staging of AMD and DR. Adjudication was provided by a second expert examiner in case of disagreement between the AI software and the reference examiner. Statistical analysis was performed on eye-level and on patient-level, by summarizing the individual image level-gradings into and eye-level or patient-level score, respectively. The performance of the RetCAD system was measured using receiver operating characteristics (ROC) analysis and sensitivity and specificity for both AMD and DR were reported. RESULTS: The RetCAD achieved an area under the ROC (Az) of 0.926 with a sensitivity of 84.6% at a specificity of 84.0% for image quality. On image level, the RetCAD software achieved Az values of 0.964 and 0.961 with sensitivity/specificity pairs of 98.2%/79.1% and 83.9%/93.3% for AMD and DR, respectively. On patient level, the RetCAD software achieved Az values of 0.960 and 0.948 with sensitivity/specificity pairs of 97.3%/73.3% and 80.0%/90.1% for AMD and DR, respectively. After adjudication by the second expert examiner sensitivity/specificity increases on patient-level to 98.6%/78.3% and 100.0%/92.3% for AMD and DR, respectively. CONCLUSION: The RetCAD offers very good sensitivity and specificity compared to manual grading by experts and is in line with that obtained by similar automated grading systems. The RetCAD AI software enables simultaneous grading of both AMD and DR based on the same fundus photos. Its sensitivity may be adjusted according to the desired acceptable sensitivity and specificity. Its simplicity cloud base integration allows cost-effective screening where routine expert evaluation may be limited.
