Evaluation of the performance of a 34-layer ResNet model-based artificial intelligence application in the diagnosis of skin diseases / 中华皮肤科杂志

ABSTRACT

Objective:To evaluate the performance of Autoderm, an artificial intelligence application, in the diagnosis of skin diseases in Chinese patients.Methods:Totally, 920 patients with confirmed skin diseases were prospectively recruited in the Department of Dermatology, the First Affiliated Hospital of Zhengzhou University. Every patient provided 1 clinical image, which was uploaded onto the Autoderm application for the diagnosis of skin diseases. The diagnostic sensitivity, specificity and accuracy of the Autoderm application were estimated, and the kappa values for the diagnostic agreement between the Autoderm application and dermatologists were calculated.Results:Among the 920 patients, 871 (94.7%) could be diagnosed with an Autoderm′s in-distribution skin disease, whereas 49 (5.3%) had out-of-distribution skin diseases. According to the top 1 and 3 diagnoses given by the Autoderm application for the 920 patients separately, its mean diagnostic sensitivities were 41.8% and 65.8%, mean specificities 96.8% and 91.5%, and mean accuracies 92.9% and 89.9%, respectively, and there was moderate overall agreement between the Autoderm application and dermatologists (κ = 0.420, 0.464, respectively). However, for an out-of-distribution skin disease, the Autoderm application could output 5 definitely false diagnoses.Conclusion:Autoderm may be used as as clinical decision support tool for the diagnosis of common skin diseases in most Chinese patients, with moderate diagnostic sensitivity, high specificity, and high accuracy, but misdiagnosis may occur.