Towards a Better Performance in Facial Expression Recognition: A Data-Centric Approach.

Facial expression is the best evidence of our emotions. Its automatic detection and recognition are key for robotics, medicine, healthcare, education, psychology, sociology, marketing, security, entertainment, and many other areas. Experiments in the lab environments achieve high performance. However, in real-world scenarios, it is challenging. Deep learning techniques based on convolutional neural networks (CNNs) have shown great potential. Most of the research is exclusively model-centric, searching for better algorithms to improve recognition. However, progress is insufficient. Despite being the main resource for automatic learning, few works focus on improving the quality of datasets. We propose a novel data-centric method to tackle misclassification, a problem commonly encountered in facial image datasets. The strategy is to progressively refine the dataset by successive training of a CNN model that is fixed. Each training uses the facial images corresponding to the correct predictions of the previous training, allowing the model to capture more distinctive features of each class of facial expression. After the last training, the model performs automatic reclassification of the whole dataset. Unlike other similar work, our method avoids modifying, deleting, or augmenting facial images. Experimental results on three representative datasets proved the effectiveness of the proposed method, improving the validation accuracy by 20.45%, 14.47%, and 39.66%, for FER2013, NHFI, and AffectNet, respectively. The recognition rates on the reclassified versions of these datasets are 86.71%, 70.44%, and 89.17% and become state-of-the-art performance.

sSLAM: Speeded-Up Visual SLAM Mixing Artificial Markers and Temporary Keypoints.

Environment landmarks are generally employed by visual SLAM (vSLAM) methods in the form of keypoints. However, these landmarks are unstable over time because they belong to areas that tend to change, e.g., shadows or moving objects. To solve this, some other authors have proposed the combination of keypoints and artificial markers distributed in the environment so as to facilitate the tracking process in the long run. Artificial markers are special elements (similar to beacons) that can be permanently placed in the environment to facilitate tracking. In any case, these systems keep a set of keypoints that is not likely to be reused, thus unnecessarily increasing the computing time required for tracking. This paper proposes a novel visual SLAM approach that efficiently combines keypoints and artificial markers, allowing for a substantial reduction in the computing time and memory required without noticeably degrading the tracking accuracy. In the first stage, our system creates a map of the environment using both keypoints and artificial markers, but once the map is created, the keypoints are removed and only the markers are kept. Thus, our map stores only long-lasting features of the environment (i.e., the markers). Then, for localization purposes, our algorithm uses the marker information along with temporary keypoints created just in the time of tracking, which are removed after a while. Since our algorithm keeps only a small subset of recent keypoints, it is faster than the state-of-the-art vSLAM approaches. The experimental results show that our proposed sSLAM compares favorably with ORB-SLAM2, ORB-SLAM3, OpenVSLAM and UcoSLAM in terms of speed, without statistically significant differences in accuracy.

Designing a cyber-physical robotic platform to assist speech-language pathologists.

This research focused on the development of a cyber-physical robotic platform to assist speech-language pathologists who are related to articulation disorders in education environments. The first goal was the design and development of the system. The second goal was the qualitative initial validation of the platform with robotics and mobile device functionalities drawing on the participation of real patients and speech-language pathologists (SLP). The research method is based on qualitative data. The first phase was the application of engineering requirements and agile techniques to build the robotic system. The second phase was to execute test sessions of the robotic platform with speech-language pathologists supervision and analyzing the experience of real male and female patients collecting data by in-depth interviews and video recordings at Heredia Special Education Center in Costa Rica. The practical approach of the cyber-physical platform has preliminarily support. Testing with SLPs and 3 other older individuals suggests that it may become a useful tool to assist professionals in the treatment of some types of articulation disorders. The time savings and data collection possibilities should be included in future investigations of efficacy.

Artificial Intelligence-based System for Detecting Attention Levels in Students.

The attention level of students in a classroom can be improved through the use of Artificial Intelligence (AI) techniques. By automatically identifying the attention level, teachers can employ strategies to regain students' focus. This can be achieved through various sources of information. One source is to analyze the emotions reflected on students' faces. AI can detect emotions, such as neutral, disgust, surprise, sadness, fear, happiness, and anger. Additionally, the direction of the students' gaze can also potentially indicate their level of attention. Another source is to observe the students' body posture. By using cameras and deep learning techniques, posture can be analyzed to determine the level of attention. For example, students who are slouching or resting their heads on their desks may have a lower level of attention. Smartwatches distributed to the students can provide biometric and other data, including heart rate and inertial measurements, which can also be used as indicators of attention. By combining these sources of information, an AI system can be trained to identify the level of attention in the classroom. However, integrating the different types of data poses a challenge that requires creating a labeled dataset. Expert input and existing studies are consulted for accurate labeling. In this paper, we propose the integration of such measurements and the creation of a dataset and a potential attention classifier. To provide feedback to the teacher, we explore various methods, such as smartwatches or direct computers. Once the teacher becomes aware of attention issues, they can adjust their teaching approach to re-engage and motivate the students. In summary, AI techniques can automatically identify the students' attention level by analyzing their emotions, gaze direction, body posture, and biometric data. This information can assist teachers in optimizing the teaching-learning process.

UrOAC: Urban objects in any-light conditions.

In the past years, several works on urban object detection from the point of view of a person have been made. These works are intended to provide an enhanced understanding of the environment for blind and visually challenged people. The mentioned approaches mostly rely in deep learning and machine learning methods. Nonetheless, these approaches only work with direct and bright light, namely, they will only perform correctly on daylight conditions. This is because deep learning algorithms require large amounts of data and the currently available datasets do not address this matter. In this work, we propose UrOAC, a dataset of urban objects captured in a range of different lightning conditions, from bright daylight to low and poor night-time lighting conditions. In the latter, the objects are only lit by low ambient light, street lamps and headlights of passing-by vehicles. The dataset depicts the following objects: pedestrian crosswalks, green traffic lights and red traffic lights. The annotations include the category and the bounding-box of each object. This dataset could be used for improve the performance at night-time and under low-light conditions of any vision-based method that involves urban objects. For instance, guidance and object detection devices for the visually challenged or self-driving and intelligent vehicles.

Accurate Multilevel Classification for Wildlife Images.

The most common approaches for classification rely on the inference of a specific class. However, every category could be naturally organized within a taxonomic tree, from the most general concept to the specific element, and that is how human knowledge works. This representation avoids the necessity of learning roughly the same features for a range of very similar categories, and it is easier to understand and work with and provides a classification for each abstraction level. In this paper, we carry out an exhaustive study of different methods to perform multilevel classification applied to the task of classifying wild animals and plant species. Different convolutional backbones, data setups, and ensembling techniques are explored to find the model which provides the best performance. As our experimentation remarks, in order to achieve the best performance on the datasets that are arranged in a tree-like structure, the classifier must feature an EfficientNetB5 backbone with an input size of 300 × 300 px, followed by a multilevel classifier. In addition, a Multiscale Crop data augmentation process must be carried out. Finally, the accuracy of this setup is a 62% top-1 accuracy and 88% top-5 accuracy. The architecture could benefit for an accuracy boost if it is involved in an ensemble of cascade classifiers, but the computational demand is unbearable for any real application.

An sEMG-Controlled 3D Game for Rehabilitation Therapies: Real-Time Time Hand Gesture Recognition Using Deep Learning Techniques.

In recent years the advances in Artificial Intelligence (AI) have been seen to play an important role in human well-being, in particular enabling novel forms of human-computer interaction for people with a disability. In this paper, we propose a sEMG-controlled 3D game that leverages a deep learning-based architecture for real-time gesture recognition. The 3D game experience developed in the study is focused on rehabilitation exercises, allowing individuals with certain disabilities to use low-cost sEMG sensors to control the game experience. For this purpose, we acquired a novel dataset of seven gestures using the Myo armband device, which we utilized to train the proposed deep learning model. The signals captured were used as an input of a Conv-GRU architecture to classify the gestures. Further, we ran a live system with the participation of different individuals and analyzed the neural network's classification for hand gestures. Finally, we also evaluated our system, testing it for 20 rounds with new participants and analyzed its results in a user study.

Biomarker Localization From Deep Learning Regression Networks.

Biomarker estimation methods from medical images have traditionally followed a segment-and-measure strategy. Deep-learning regression networks have changed such a paradigm, enabling the direct estimation of biomarkers in databases where segmentation masks are not present. While such methods achieve high performance, they operate as a black-box. In this work, we present a novel deep learning network structure that, when trained with only the value of the biomarker, can perform biomarker regression and the generation of an accurate localization mask simultaneously, thus enabling a qualitative assessment of the image locus that relates to the quantitative result. We showcase the proposed method with three different network structures and compare their performance against direct regression networks in four different problems: pectoralis muscle area (PMA), subcutaneous fat area (SFA), liver mass area in single slice computed tomography (CT), and Agatston score estimated from non-contrast thoracic CT images (CAC). Our results show that the proposed method improves the performance with respect to direct biomarker regression methods (correlation coefficient of 0.978, 0.998, and 0.950 for the proposed method in comparison to 0.971, 0.982, and 0.936 for the reference regression methods on PMA, SFA and CAC respectively) while achieving good localization (DICE coefficients of 0.875, 0.914 for PMA and SFA respectively, p < 0.05 for all pairs). We observe the same improvement in regression results comparing the proposed method with those obtained by quantify the outputs using an U-Net segmentation network (0.989 and 0.951 respectively). We, therefore, conclude that it is possible to obtain simultaneously good biomarker regression and localization when training biomarker regression networks using only the biomarker value.

PHAROS 2.0-A PHysical Assistant RObot System Improved.

There are great physical and cognitive benefits for older adults who are engaged in active aging, a process that should involve daily exercise. In our previous work on the PHysical Assistant RObot System (PHAROS), we developed a system that proposed and monitored physical activities. The system used a social robot to analyse, by means of computer vision, the exercise a person was doing. Then, a recommender system analysed the exercise performed and indicated what exercise to perform next. However, the system needed certain improvements. On the one hand, the vision system captured the movement of the person and indicated whether the exercise had been done correctly or not. On the other hand, the recommender system was based purely on a ranking system that did not take into account temporal evolution and preferences. In this work, we propose an evolution of PHAROS, PHAROS 2.0, incorporating improvements in both of the previously mentioned aspects. In the motion capture aspect, we are now able to indicate the degree of completeness of each exercise, identifying the part that has not been done correctly, and a real-time performance correction. In this way, the recommender system receives a greater amount of information and so can more accurately indicate the exercise to be performed. In terms of the recommender system, an algorithm was developed to weigh the performance, temporal evolution and preferences, providing a more accurate recommendation, as well as expanding the recommendation to a batch of exercises, instead of just one.

UASOL, a large-scale high-resolution outdoor stereo dataset.

In this paper, we propose a new dataset for outdoor depth estimation from single and stereo RGB images. The dataset was acquired from the point of view of a pedestrian. Currently, the most novel approaches take advantage of deep learning-based techniques, which have proven to outperform traditional state-of-the-art computer vision methods. Nonetheless, these methods require large amounts of reliable ground-truth data. Despite there already existing several datasets that could be used for depth estimation, almost none of them are outdoor-oriented from an egocentric point of view. Our dataset introduces a large number of high-definition pairs of color frames and corresponding depth maps from a human perspective. In addition, the proposed dataset also features human interaction and great variability of data, as shown in this work.

Rehabilitation Technology: Assistance from Hospital to Home.

Rehabilitation is essential for disabled people to achieve the highest level of functional independence, reducing or preventing impairments. Nonetheless, this process can be long and expensive. This fact together with the ageing phenomenon has become a critical issue for both clinicians and patients. In this sense, technological solutions may be beneficial since they reduce the costs and increase the number of patients per caregiver, which makes them more accessible. In addition, they provide access to rehabilitation services for those facing physical, financial, and/or attitudinal barriers. This paper presents the state of the art of the assistive rehabilitation technologies for different recovery methods starting from in-person sessions to complementary at-home activities.

Enhancing the Ambient Assisted Living Capabilities with a Mobile Robot.

Ambient assisted living (AAL) environments are currently a key focus of interest as an option to assist and monitor disabled and elderly people. These systems can improve their quality of life and personal autonomy by detecting events such as entering potentially dangerous areas, potential fall events, or extended stays in the same place. Nonetheless, there are areas that remain outside the scope of AAL systems due to the placement of cameras. There also exist sources of danger in the scope of the camera that the AAL system cannot detect. These sources of danger are relatively small in size, occluded, or nonstatic. To solve this problem, we propose the inclusion of a robot which maps such uncovered areas looking for new potentially dangerous areas that go unnoticed by the AAL. The robot then sends this information to the AAL system in order to improve its performance. Experimentation in real-life scenarios successfully validates our approach.

Inferring Static Hand Poses from a Low-Cost Non-Intrusive sEMG Sensor.

Every year, a significant number of people lose a body part in an accident, through sickness or in high-risk manual jobs. Several studies and research works have tried to reduce the constraints and risks in their lives through the use of technology. This work proposes a learning-based approach that performs gesture recognition using a surface electromyography-based device, the Myo Armband released by Thalmic Labs, which is a commercial device and has eight non-intrusive low-cost sensors. With 35 able-bodied subjects, and using the Myo Armband device, which is able to record data at about 200 MHz, we collected a dataset that includes six dissimilar hand gestures. We used a gated recurrent unit network to train a system that, as input, takes raw signals extracted from the surface electromyography sensors. The proposed approach obtained a 99.90% training accuracy and 99.75% validation accuracy. We also evaluated the proposed system on a test set (new subjects) obtaining an accuracy of 77.85%. In addition, we showed the test prediction results for each gesture separately and analyzed which gestures for the Myo armband with our suggested network can be difficult to distinguish accurately. Moreover, we studied for first time the gated recurrent unit network capability in gesture recognition approaches. Finally, we integrated our method in a system that is able to classify live hand gestures.

LOCALIZING IMAGE-BASED BIOMARKER REGRESSION WITHOUT TRAINING MASKS: A NEW APPROACH TO BIOMARKER DISCOVERY.

Biomarker inference from biomedical images is one of the main tasks of medical image analysis. Standard techniques follow a segmentation-and-measure strategy, where the structure is first segmented and then the measurement is performed. Recent work has shown that such strategy could be replaced by a direct regression of the biomarker value in using regression networks. While achieving high correlation coefficients, such techniques operate as a 'black-box', not offering quality-control images. We present a methodology to regress the biomarker from the image while simultaneously computing the quality control image. Our proposed methodology does not require segmentation masks for training, but infers the segmentations directly from the pixels that used to compute the biomarker value. The network proposed consists of two steps: a segmentation method to an unknown reference and a summation method for the biomarker estimation. The network is optimized using a dual loss function, L2 for the biomarkers and an L1 to enforce sparsity. We showcase our methodology in the problem of pectoralis muscle area (PMA) and subcutaneous fat area (SFA) inference in a single slice from chest-CT images. We use a database of 7000 cases to which only the value of the biomarker is known for training and a test set of 3000 cases with both, biomarkers and segmentations. We achieve a correlation coefficient of 0.97 for PMA and 0.98 for SFA with respect to the reference standard. The average DICE coefficient is of 0.88 (PMA) and 0.89 (SFA). Comparing with standard segment-and-measure techniques, we achieve the same correlation for the biomarkers but smaller DICE coefficients in segmentation. Such is of little surprise, since segmentation networks are the upper limit of performance achievable, and we are not using segmentation masks for training. We can conclude that it is possible to infer segmentation masks from biomarker regression networks.

PHAROS-PHysical Assistant RObot System.

The great demographic change leading to an ageing society demands technological solutions to satisfy the increasing varied elderly needs. This paper presents PHAROS, an interactive robot system that recommends and monitors physical exercises designed for the elderly. The aim of PHAROS is to be a friendly elderly companion that periodically suggests personalised physical activities, promoting healthy living and active ageing. Here, it is presented the PHAROS architecture, components and experimental results. The architecture has three main strands: a Pepper robot, that interacts with the users and records their exercises performance; the Human Exercise Recognition, that uses the Pepper recorded information to classify the exercise performed using Deep Leaning methods; and the Recommender, a smart-decision maker that schedules periodically personalised physical exercises in the users' agenda. The experimental results show a high accuracy in terms of detecting and classifying the physical exercises (97.35%) done by 7 persons. Furthermore, we have implemented a novel procedure of rating exercises on the recommendation algorithm. It closely follows the users' health status (poor performance may reveal health problems) and adapts the suggestions to it. The history may be used to access the physical condition of the user, revealing underlying problems that may be impossible to see otherwise.

Automated Agatston Score Computation in non-ECG Gated CT Scans Using Deep Learning.

INTRODUCTION: The Agatston score is a well-established metric of cardiovascular disease related to clinical outcomes. It is computed from CT scans by a) measuring the volume and intensity of the atherosclerotic plaques and b) aggregating such information in an index. OBJECTIVE: To generate a convolutional neural network that inputs a non-contrast chest CT scan and outputs the Agatston score associated with it directly, without a prior segmentation of Coronary Artery Calcifications (CAC). MATERIALS AND METHODS: We use a database of 5973 non-contrast non-ECG gated chest CT scans where the Agatston score has been manually computed. The heart of each scan is cropped automatically using an object detector. The database is split in 4973 cases for training and 1000 for testing. We train a 3D deep convolutional neural network to regress the Agatston score directly from the extracted hearts. RESULTS: The proposed method yields a Pearson correlation coefficient of r = 0.93; p ≤ 0.0001 against manual reference standard in the 1000 test cases. It further stratifies correctly 72.6% of the cases with respect to standard risk groups. This compares to more complex state-of-the-art methods based on prior segmentations of the CACs, which achieve r = 0.94 in ECG-gated pulmonary CT. CONCLUSIONS: A convolutional neural network can regress the Agatston score from the image of the heart directly, without a prior segmentation of the CACs. This is a new and simpler paradigm in the Agatston score computation that yields similar results to the state-of-the-art literature.

On the Relevance of the Loss Function in the Agatston Score Regression from Non-ECG Gated CT Scans.

In this work, we evaluate the relevance of the choice of loss function in the regression of the Agatston score from 3D heart volumes obtained from non-contrast non-ECG gated chest computed tomography scans. The Agatston score is a well-established metric of cardiovascular disease, where an index of coronary artery disease (CAD) is computed by segmenting the calcifications of the arteries and multiplying each calcification by a factor related to their intensity and their volume, creating a final aggregated index. Recent work has automated such task with deep learning techniques, even skipping the segmentation step and performing a direct regression of the Agatston score. We study the effect of the choice of the loss function in such methodologies. We use a large database of 6983 CT scans to which the Agatston score has been manually computed. The dataset is split into a training set and a validation set of n = 1000. We train a deep learning regression network using such data with different loss functions while keeping the structure of the network and training parameters constant. Pearson correlation coefficient ranges from 0.902 to 0.938 depending on the loss function. Correct risk group assignment measurements range between 59.5% and 81.7%. There is a trade-off between the accuracy of the Pearson correlation coefficient and the risk group measurement, which leads to optimize for one or the other.

Geoffrey: An Automated Schedule System on a Social Robot for the Intellectually Challenged.

The accelerated growth of the percentage of elder people and persons with brain injury-related conditions and who are intellectually challenged are some of the main concerns of the developed countries. These persons often require special cares and even almost permanent overseers that help them to carry out diary tasks. With this issue in mind, we propose an automated schedule system which is deployed on a social robot. The robot keeps track of the tasks that the patient has to fulfill in a diary basis. When a task is triggered, the robot guides the patient through its completion. The system is also able to detect if the steps are being properly carried out or not, issuing alerts in that case. To do so, an ensemble of deep learning techniques is used. The schedule is customizable by the carers and authorized relatives. Our system could enhance the quality of life of the patients and improve their self-autonomy. The experimentation, which was supervised by the ADACEA foundation, validates the achievement of these goals.

A comparative study of registration methods for RGB-D video of static scenes.

The use of RGB-D sensors for mapping and recognition tasks in robotics or, in general, for virtual reconstruction has increased in recent years. The key aspect of these kinds of sensors is that they provide both depth and color information using the same device. In this paper, we present a comparative analysis of the most important methods used in the literature for the registration of subsequent RGB-D video frames in static scenarios. The analysis begins by explaining the characteristics of the registration problem, dividing it into two representative applications: scene modeling and object reconstruction. Then, a detailed experimentation is carried out to determine the behavior of the different methods depending on the application. For both applications, we used standard datasets and a new one built for object reconstruction.

Topological visual mapping in robotics.

A key problem in robotics is the construction of a map from its environment. This map could be used in different tasks, like localization, recognition, obstacle avoidance, etc. Besides, the simultaneous location and mapping (SLAM) problem has had a lot of interest in the robotics community. This paper presents a new method for visual mapping, using topological instead of metric information. For that purpose, we propose prior image segmentation into regions in order to group the extracted invariant features in a graph so that each graph defines a single region of the image. Although others methods have been proposed for visual SLAM, our method is complete, in the sense that it makes all the process: it presents a new method for image matching; it defines a way to build the topological map; and it also defines a matching criterion for loop-closing. The matching process will take into account visual features and their structure using the graph transformation matching (GTM) algorithm, which allows us to process the matching and to remove out the outliers. Then, using this image comparison method, we propose an algorithm for constructing topological maps. During the experimentation phase, we will test the robustness of the method and its ability constructing topological maps. We have also introduced new hysteresis behavior in order to solve some problems found building the graph.