Interpretation and explanation of computer vision classification of carambola (Averrhoa carambola L.) according to maturity stage.

ABSTRACT

The classification of carambola, also known as starfruit, according to quality parameters is usually conducted by trained human evaluators through visual inspections. This is a costly and subjective method that can generate high variability in results. As an alternative, computer vision systems (CVS) combined with deep learning (DCVS) techniques have been introduced in the industry as a powerful and an innovative tool for the rapid and non-invasive classification of fruits. However, validating the learning capability and trustworthiness of a DL model, aka black box, to obtain insights can be challenging. To reduce this gap, we propose an integrated eXplainable Artificial Intelligence (XAI) method for the classification of carambolas at different maturity stages. We compared two Residual Neural Networks (ResNet) and Visual Transformers (ViT) to identify the image regions that are enhanced by a Random Forest (RF) model, with the aim of providing more detailed information at the feature level for classifying the maturity stage. Changes in fruit colour and physicochemical data throughout the maturity stages were analysed, and the influence of these parameters on the maturity stages was evaluated using the Gradient-weighted Class Activation Mapping (Grad-CAM), the Attention Maps using RF importance. The proposed approach provides a visualization and description of the most important regions that led to the model decision, in wide visualization follows the models an importance features from RF. Our approach has promising potential for standardized and rapid carambolas classification, achieving 91 % accuracy with ResNet and 95 % with ViT, with potential application for other fruits.