RESUMO
Despite the success of hand-crafted features in computer visioning for many years, nowadays, this has been replaced by end-to-end learnable features that are extracted from deep convolutional neural networks (CNNs). Whilst CNNs can learn robust features directly from image pixels, they require large amounts of samples and extreme augmentations. On the contrary, hand-crafted features, like SIFT, exhibit several interesting properties as they can provide local rotation invariance. In this work, a novel scheme combining the strengths of SIFT descriptors with CNNs, namely SIFT-CNN, is presented. Given a single-channel image, one SIFT descriptor is computed for every pixel, and thus, every pixel is represented as an M-dimensional histogram, which ultimately results in an M-channel image. Thus, the SIFT image is generated from the SIFT descriptors for all the pixels in a single-channel image, while at the same time, the original spatial size is preserved. Next, a CNN is trained to utilize these M-channel images as inputs by operating directly on the multiscale SIFT images with the regular convolution processes. Since these images incorporate spatial relations between the histograms of the SIFT descriptors, the CNN is guided to learn features from local gradient information of images that otherwise can be neglected. In this manner, the SIFT-CNN implicitly acquires a local rotation invariance property, which is desired for problems where local areas within the image can be rotated without affecting the overall classification result of the respective image. Some of these problems refer to indirect immunofluorescence (IIF) cell image classification, ground-based all-sky image-cloud classification and human lip-reading classification. The results for the popular datasets related to the three different aforementioned problems indicate that the proposed SIFT-CNN can improve the performance and surpasses the corresponding CNNs trained directly on pixel values in various challenging tasks due to its robustness in local rotations. Our findings highlight the importance of the input image representation in the overall efficiency of a data-driven system.
RESUMO
Human assistive technology and computer-aided diagnosis is an emerging field in the area of medical imaging. Following the recent advances in this domain, a study for integrating machine learning techniques in musculoskeletal ultrasonography images was conducted. The goal of this attempt was to investigate how feature extraction techniques, that capture higher-level information, perform in identifying human characteristics. The potential success of these techniques could lead to significant improvement of the current assessment methods-as the gray-scale image analysis-for distinguishing healthy and pathologic conditions, that are heavily dependent on the image-acquisition system. The contribution of this work is threefold. First, a new privately held data set of 74 healthy patients was presented. This data set included musculoskeletal ultrasound images from four muscles of the human body, namely the biceps brachii, tibialis anterior, gastrocnemius medialis and rectus femoris, recorded in the transverse and longitudinal plane. Second, two classification tasks were performed, namely, gender and muscle-type recognition, to assess the performance of the proposed method for successfully identifying differences in the texture of the examined muscle sections. Third, a novel method used with great success in the computer vision domain was presented, allowing the extraction of a high-level feature representation, by encoding the distribution of locally invariant texture descriptors. On the muscle-type recognition our method achieved an 87.07% classification rate, and on the task of gender recognition it surpassed state-of-the-art textural representations, reported in the literature in almost all the examined muscle sections.
