ABSTRACT
Introduction: The use of deep convolutional neural networks for analyzing skin lesion images has shown promising results. The identification of skin cancer by faster and less expensive means can lead to an early diagnosis, saving lives and avoiding treatment costs. However, to implement this technology in a clinical context, it is important for specialists to understand why a certain model makes a prediction; it must be explainable. Explainability techniques can be used to highlight the patterns of interest for a prediction. Methods: Our goal was to test five different techniques: Grad-CAM, Grad-CAM++, Score-CAM, Eigen-CAM, and LIME, to analyze the agreement rate between features highlighted by the visual explanation maps to 3 important clinical criteria for melanoma classification: asymmetry, border irregularity, and color heterogeneity (ABC rule) in 100 melanoma images. Two dermatologists scored the visual maps and the clinical images using a semi-quantitative scale, and the results were compared. They also ranked their preferable techniques. Results: We found that the techniques had different agreement rates and acceptance. In the overall analysis, Grad-CAM showed the best total+partial agreement rate (93.6%), followed by LIME (89.8%), Grad-CAM++ (88.0%), Eigen-CAM (86.4%), and Score-CAM (84.6%). Dermatologists ranked their favorite options: Grad-CAM and Grad-CAM++, followed by Score-CAM, LIME, and Eigen-CAM. Discussion: Saliency maps are one of the few methods that can be used for visual explanations. The evaluation of explainability with humans is ideal to assess the understanding and applicability of these methods. Our results demonstrated that there is a significant agreement between clinical features used by dermatologists to diagnose melanomas and visual explanation techniques, especially Grad-Cam.
ABSTRACT
Skin cancer is currently the most common type of cancer among Caucasians. The increase in life expectancy, along with new diagnostic tools and treatments for skin cancer, has resulted in unprecedented changes in patient care and has generated a great burden on healthcare systems. Early detection of skin tumors is expected to reduce this burden. Artificial intelligence (AI) algorithms that support skin cancer diagnoses have been shown to perform at least as well as dermatologists' diagnoses. Recognizing the need for clinically and economically efficient means of diagnosing skin cancers at early stages in the primary care attention, we developed an efficient computer-aided diagnosis (CAD) system to be used by primary care physicians (PCP). Additionally, we developed a smartphone application with a protocol for data acquisition (i.e., photographs, demographic data and short clinical histories) and AI algorithms for clinical and dermoscopic image classification. For each lesion analyzed, a report is generated, showing the image of the suspected lesion and its respective Heat Map; the predicted probability of the suspected lesion being melanoma or malignant; the probable diagnosis based on that probability; and a suggestion on how the lesion should be managed. The accuracy of the dermoscopy model for melanoma was 89.3%, and for the clinical model, 84.7% with 0.91 and 0.89 sensitivity and 0.89 and 0.83 specificity, respectively. Both models achieved an area under the curve (AUC) above 0.9. Our CAD system can screen skin cancers to guide lesion management by PCPs, especially in the contexts where the access to the dermatologist can be difficult or time consuming. Its use can enable risk stratification of lesions and/or patients and dramatically improve timely access to specialist care for those requiring urgent attention.
