Assessing student engagement from facial behavior in on-line learning.

The automatic monitoring and assessment of the engagement level of learners in distance education may help in understanding problems and providing personalized support during the learning process. This article presents a research aiming to investigate how student engagement level can be assessed from facial behavior and proposes a model based on Long Short-Term Memory (LSTM) networks to predict the level of engagement from facial action units, gaze, and head poses. The dataset used to learn the model is the one of the EmotiW 2019 challenge datasets. In order to test its performance in learning contexts, an experiment, involving students attending an online lecture, was performed. The aim of the study was to compare the self-evaluation of the engagement perceived by the students with the one assessed by the model. During the experiment we collected videos of students behavior and, at the end of each session, we asked students to answer a questionnaire for assessing their perceived engagement. Then, the collected videos were analyzed automatically with a software that implements the model and provides an interface for the visual analysis of the model outcome. Results show that, globally, engagement prediction from students' facial behavior was weakly correlated to their subjective answers. However, when considering only the emotional dimension of engagement, this correlation is stronger and the analysis of facial action units and head pose (facial movements) are positively correlated with it, while there is an inverse correlation with the gaze, meaning that the more the student's feels engaged the less are the gaze movements.

Adolescents with borderline personality disorder show a higher response to stress but a lack of self-perception: Evidence through affective computing.

Stress reactivity is a complex phenomenon associated with multiple and multimodal expressions and functions. Herein, we hypothesized that compared with healthy controls (HCs), adolescents with borderline personality disorder (BPD) would exhibit a stronger response to stressors and a deficit in self-perception of stress due to their lack of insight. Twenty adolescents with BPD and 20 matched HCs performed a socially evaluated mental arithmetic test to induce stress. We assessed self- and heteroperception using both human ratings and affective computing-based methods for the automatic extraction of 39 behavioral features (2D + 3D video recording) and 62 physiological features (Nexus-10 recording). Predictions were made using machine learning. In addition, salivary cortisol was measured. Human ratings showed that adolescents with BPD experienced more stress than HCs. Human ratings and automated machine learning indicated opposite results regarding self- and heteroperceived stress in adolescents with BPD compared to HCs. Adolescents with BPD had higher levels of heteroperceived stress than self-perceived stress. Similarly, affective computing achieved better classification for heteroperceived stress. HCs had an opposite profile; they had higher levels of self-perceived stress, and affective computing reached a better classification for self-perceived stress. We conclude that adolescents with BPD are more sensitive to stress and show a lack of self-perception (or insight). In terms of clinical implications, our affective computing measures may help distinguish hetero- vs. self-perceptions of stress in natural settings and may offer external feedback during therapeutic interaction.

Behavior and interaction imaging at 9 months of age predict autism/intellectual disability in high-risk infants with West syndrome.

Automated behavior analysis are promising tools to overcome current assessment limitations in psychiatry. At 9 months of age, we recorded 32 infants with West syndrome (WS) and 19 typically developing (TD) controls during a standardized mother-infant interaction. We computed infant hand movements (HM), speech turn taking of both partners (vocalization, pause, silences, overlap) and motherese. Then, we assessed whether multimodal social signals and interactional synchrony at 9 months could predict outcomes (autism spectrum disorder (ASD) and intellectual disability (ID)) of infants with WS at 4 years. At follow-up, 10 infants developed ASD/ID (WS+). The best machine learning reached 76.47% accuracy classifying WS vs. TD and 81.25% accuracy classifying WS+ vs. WS-. The 10 best features to distinguish WS+ and WS- included a combination of infant vocalizations and HM features combined with synchrony vocalization features. These data indicate that behavioral and interaction imaging was able to predict ASD/ID in high-risk children with WS.

ICT and autism care: state of the art.

PURPOSE OF REVIEW: Over the past 10 years, the use of information and communication technologies (ICTs) has increased in regard to the treatment of individuals with autism spectrum disorders (ASDs). ICT support mechanisms (e.g. computers, laptops, robots) are particularly attractive and are adapted to children with ASD. In addition, ICT algorithms can offer new perspectives for clinicians, outside direct apps or gaming proposals. Here, we will focus on the use of serious games and robots because of their attractiveness and their value in working on social skills. RECENT FINDINGS: The latest knowledge regarding the use of ICT in the forms of serious games and robotics applied to individuals with ASD shows that the field of serious games has already achieved interesting and promising results, although the clinical validations are not always complete. In the field of robotics, there are still many limitations on the use of ICT (e.g. most interaction are similar to the wizard of Oz), and questions remain concerning their eventual effectiveness. SUMMARY: To describe the implications of the findings for clinical practice or research, we describe two large projects, namely, JEMImE and Michelangelo, as examples of current studies that are aimed at enhancing social skills in children with ASD by including novel algorithms with clinical insights in robots or serious games.