Establishing and validating a spotted tongue recognition and extraction model based on multiscale convolutional neural network / 数字中医药（英文）

ABSTRACT

@#Objective In tongue diagnosis, the location, color, and distribution of spots can be used to speculate on the viscera and severity of the heat evil. This work focuses on the image analysis method of artificial intelligence (AI) to study the spotted tongue recognition of traditional Chinese medicine (TCM). Methods A model of spotted tongue recognition and extraction is designed, which is based on the principle of image deep learning and instance segmentation. This model includes multiscale feature map generation, region proposal searching, and target region recognition. Firstly, deep convolution network is used to build multiscale low- and high-abstraction feature maps after which, target candidate box generation algorithm and selection strategy are used to select high-quality target candidate regions. Finally, classification network is used for classifying target regions and calculating target region pixels. As a result, the region segmentation of spotted tongue is obtained. Under non-standard illumination conditions, various tongue images were taken by mobile phones, and experiments were conducted. Results The spotted tongue recognition achieved an area under curve (AUC) of 92.40%, an accuracy of 84.30% with a sensitivity of 88.20%, a specificity of 94.19%, a recall of 88.20%, a regional pixel accuracy index pixel accuracy (PA) of 73.00%, a mean pixel accuracy (mPA) of 73.00%, an intersection over union (IoU) of 60.00%, and a mean intersection over union (mIoU) of 56.00%. Conclusion The results of the study verify that the model is suitable for the application of the TCM tongue diagnosis system. Spotted tongue recognition via multiscale convolutional neural network (CNN) would help to improve spot classification and the accurate extraction of pixels of spot area as well as provide a practical method for intelligent tongue diagnosis of TCM.