eSEE-d: Emotional State Estimation Based on Eye-Tracking Dataset.

Affective state estimation is a research field that has gained increased attention from the research community in the last decade. Two of the main catalysts for this are the advancement in the data analysis using artificial intelligence and the availability of high-quality video. Unfortunately, benchmarks and public datasets are limited, thus making the development of new methodologies and the implementation of comparative studies essential. The current work presents the eSEE-d database, which is a resource to be used for emotional State Estimation based on Eye-tracking data. Eye movements of 48 participants were recorded as they watched 10 emotion-evoking videos, each of them followed by a neutral video. Participants rated four emotions (tenderness, anger, disgust, sadness) on a scale from 0 to 10, which was later translated in terms of emotional arousal and valence levels. Furthermore, each participant filled three self-assessment questionnaires. An extensive analysis of the participants' answers to the questionnaires' self-assessment scores as well as their ratings during the experiments is presented. Moreover, eye and gaze features were extracted from the low-level eye-recorded metrics, and their correlations with the participants' ratings are investigated. Finally, we take on the challenge to classify arousal and valence levels based solely on eye and gaze features, leading to promising results. In particular, the Deep Multilayer Perceptron (DMLP) network we developed achieved an accuracy of 92% in distinguishing positive valence from non-positive and 81% in distinguishing low arousal from medium arousal. The dataset is made publicly available.

Review of Eye Tracking Metrics Involved in Emotional and Cognitive Processes.

Eye behaviour provides valuable information revealing one's higher cognitive functions and state of affect. Although eye tracking is gaining ground in the research community, it is not yet a popular approach for the detection of emotional and cognitive states. In this paper, we present a review of eye and pupil tracking related metrics (such as gaze, fixations, saccades, blinks, pupil size variation, etc.) utilized towards the detection of emotional and cognitive processes, focusing on visual attention, emotional arousal and cognitive workload. Besides, we investigate their involvement as well as the computational recognition methods employed for the reliable emotional and cognitive assessment. The publicly available datasets employed in relevant research efforts were collected and their specifications and other pertinent details are described. The multimodal approaches which combine eye-tracking features with other modalities (e.g. biosignals), along with artificial intelligence and machine learning techniques were also surveyed in terms of their recognition/classification accuracy. The limitations, current open research problems and prospective future research directions were discussed for the usage of eye-tracking as the primary sensor modality. This study aims to comprehensively present the most robust and significant eye/pupil metrics based on available literature towards the development of a robust emotional or cognitive computational model.

A User-Centric approach for Personalization based on Human Activity Recognition.

The objective of this work focuses on multiple independent user profiles that capture behavioral, emotional, medical, and physical patterns in the working and living environment resulting in one general user profile. Depending on the user's current activity (e.g. walking, eating, etc.), medical history, and other influential factors, the developed framework acts as a supplemental assistant to the user by providing not only the ability to enable supportive functionalities (e.g. image filtering, magnification, etc.) but also informative recommendations (e.g. diet, alcohol, etc.). The personalization of such a profile lies within the user's past preferences using human activity recognition as a base, and it is achieved through a statistical model, the Bayesian belief network. Training and real-time methodological pipelines are introduced and validated. The employed deep learning techniques for identifying human activities are presented and validated in publicly available and in-house datasets. The overall accuracy of human activity recognition reaches up to 86.96 %.

COLET: A dataset for COgnitive workLoad estimation based on eye-tracking.

BACKGROUND AND OBJECTIVE: The cognitive workload is an important component in performance psychology, ergonomics, and human factors. Publicly available datasets are scarce, making it difficult to establish new approaches and comparative studies. In this work, COLET-COgnitive workLoad estimation based on Eye-Tracking dataset is presented. METHODS: Forty-seven (47) individuals' eye movements were monitored as they solved puzzles involving visual search activities of varying complexity and duration. The participants' cognitive workload level was evaluated with the subjective test of NASA-TLX and this score is used as an annotation of the activity. Extensive data analysis was performed in order to derive eye and gaze features from low-level eye recorded metrics, and a range of machine learning models were evaluated and tested regarding the estimation of the cognitive workload level. RESULTS: The activities induced four different levels of cognitive workload. Multi tasking and time pressure have induced a higher level of cognitive workload than the one induced by single tasking and absence of time pressure. Multi tasking had a significant effect on 17 eye features while time pressure had a significant effect on 7 eye features. Both binary and multi-class identification attempts were performed by testing a variety of well-known classifiers, resulting in encouraging results towards cognitive workload levels estimation, with up to 88% correct predictions between low and high cognitive workload. CONCLUSIONS: Machine learning analysis demonstrated potential in discriminating cognitive workload levels using only eye-tracking characteristics. The proposed dataset includes a much higher sample size and a wider spectrum of eye and gaze metrics than other similar datasets, allowing for the examination of their relations with various cognitive states.

Real Time Human Activity Recognition Using Acceleration and First-Person Camera data.

The aim of this work is to present an automated method, working in real time, for human activity recognition based on acceleration and first-person camera data. A Long-Short-Term-Memory (LSTM) model has been built for recognizing locomotive activities (i.e. walking, sitting, standing, going upstairs, going downstairs) from acceleration data, while a ResNet model is employed for the recognition of stationary activities (i.e. eating, reading, writing, watching TV working on PC). The outcomes of the two models are fused in order for the final decision, regarding the performed activity, to be made. For the training, testing and evaluation of the proposed models, a publicly available dataset and an "in-house" dataset are utilized. The overall accuracy of the proposed algorithmic pipeline reaches 87.8%.

Realization of a service for the long-term risk assessment of diabetes-related complications.

AIM: We present a computerized system for the assessment of the long-term risk of developing diabetes-related complications. METHODS: The core of the system consists of a set of predictive models, developed through a data-mining/machine-learning approach, which are able to evaluate individual patient profiles and provide personalized risk assessments. Missing data is a common issue in (electronic) patient records, thus the models are paired with a module for the intelligent management of missing information. RESULTS: The system has been deployed and made publicly available as Web service, and it has been fully integrated within the diabetes-management platform developed by the European project REACTION. Preliminary usability tests showed that the clinicians judged the models useful for risk assessment and for communicating the risk to the patient. Furthermore, the system performs as well as the United Kingdom Prospective Diabetes Study (UKPDS) Risk Engine when both systems are tested on an independent cohort of UK diabetes patients. CONCLUSIONS: Our work provides a working example of risk-stratification tool that is (a) specific for diabetes patients, (b) able to handle several different diabetes related complications, (c) performing as well as the widely known UKPDS Risk Engine on an external validation cohort.