Assessing student engagement from facial behavior in on-line learning.

The automatic monitoring and assessment of the engagement level of learners in distance education may help in understanding problems and providing personalized support during the learning process. This article presents a research aiming to investigate how student engagement level can be assessed from facial behavior and proposes a model based on Long Short-Term Memory (LSTM) networks to predict the level of engagement from facial action units, gaze, and head poses. The dataset used to learn the model is the one of the EmotiW 2019 challenge datasets. In order to test its performance in learning contexts, an experiment, involving students attending an online lecture, was performed. The aim of the study was to compare the self-evaluation of the engagement perceived by the students with the one assessed by the model. During the experiment we collected videos of students behavior and, at the end of each session, we asked students to answer a questionnaire for assessing their perceived engagement. Then, the collected videos were analyzed automatically with a software that implements the model and provides an interface for the visual analysis of the model outcome. Results show that, globally, engagement prediction from students' facial behavior was weakly correlated to their subjective answers. However, when considering only the emotional dimension of engagement, this correlation is stronger and the analysis of facial action units and head pose (facial movements) are positively correlated with it, while there is an inverse correlation with the gaze, meaning that the more the student's feels engaged the less are the gaze movements.

Automatic facial emotion recognition at the COVID-19 pandemic time.

People use various nonverbal communicative channels to convey emotions, among which facial expressions are considered the most important ones. Thus, automatic Facial Expression Recognition (FER) is a fundamental task to increase the perceptive skills of computers, especially in human-computer interaction. Like humans, state-of-art FER systems are able to recognize emotions from the entire face of a person. However, the COVID-19 pandemic has imposed a massive use of face masks that help in preventing infection but may hamper social communication and make the recognition of facial expressions a very challenging task due to facial occlusion. In this paper we propose a FER system capable to recognize emotions from masked faces. The system checks for the presence of a mask on the face image and, in case of mask detection, it extracts the eyes region and recognizes the emotion only considering that portion of the face. The effectiveness of the developed FER system was tested in recognizing emotions and their valence only from the eyes region and comparing the results when considering the entire face. As it was expected, emotions that are related mainly to the mouth region (e.g., disgust) are barely recognized, while positive emotions are better identified by considering only the eyes region. Moreover, we compared the results of our FER system to the human annotation of emotions on masked faces. We found out that the FER system outperforms the human annotation, thus showing that the model is able to learn proper features for each emotion leveraging only the eyes region.

Dancing With Parkinson's Disease: The SI-ROBOTICS Study Protocol.

Introduction: Parkinson's disease (PD) is one of the most frequent causes of disability among older people, characterized by motor disorders, rigidity, and balance problems. Recently, dance has started to be considered an effective exercise for people with PD. In particular, Irish dancing, along with tango and different forms of modern dance, may be a valid strategy to motivate people with PD to perform physical activity. The present protocol aims to implement and evaluate a rehabilitation program based on a new system called "SI-ROBOTICS," composed of multiple technological components, such as a social robotic platform embedded with an artificial vision setting, a dance-based game, environmental and wearable sensors, and an advanced AI reasoner module. Methods and Analysis: For this study, 20 patients with PD will be recruited. Sixteen therapy sessions of 50 min will be conducted (two training sessions per week, for 8 weeks), involving two patients at a time. Evaluation will be primarily focused on the acceptability of the SI-ROBOTICS system. Moreover, the analysis of the impact on the patients' functional status, gait, balance, fear of falling, cardio-respiratory performance, motor symptoms related to PD, and quality of life, will be considered as secondary outcomes. The trial will start in November 2021 and is expected to end by April 2022. Discussions: The study aims to propose and evaluate a new approach in PD rehabilitation, focused on the use of Irish dancing, together with a new technological system focused on helping the patient perform the dance steps and on collecting kinematic and performance parameters used both by the physiotherapist (for the evaluation and planning of the subsequent sessions) and by the system (to outline the levels of difficulty of the exercise). Ethics and Dissemination: The study was approved by the Ethics Committee of the IRCCS INRCA. It was recorded in ClinicalTrials.gov on the number NCT05005208. The study findings will be used for publication in peer-reviewed scientific journals and presentations in scientific meetings.