Recurrent Network Solutions for Human Posture Recognition Based on Kinect Skeletal Data.

Ambient Assisted Living (AAL) systems are designed to provide unobtrusive and user-friendly support in daily life and can be used for monitoring frail people based on various types of sensors, including wearables and cameras. Although cameras can be perceived as intrusive in terms of privacy, low-cost RGB-D devices (i.e., Kinect V2) that extract skeletal data can partially overcome these limits. In addition, deep learning-based algorithms, such as Recurrent Neural Networks (RNNs), can be trained on skeletal tracking data to automatically identify different human postures in the AAL domain. In this study, we investigate the performance of two RNN models (2BLSTM and 3BGRU) in identifying daily living postures and potentially dangerous situations in a home monitoring system, based on 3D skeletal data acquired with Kinect V2. We tested the RNN models with two different feature sets: one consisting of eight human-crafted kinematic features selected by a genetic algorithm, and another consisting of 52 ego-centric 3D coordinates of each considered skeleton joint, plus the subject's distance from the Kinect V2. To improve the generalization ability of the 3BGRU model, we also applied a data augmentation method to balance the training dataset. With this last solution we reached an accuracy of 88%, the best we achieved so far.

Neural Networks for Automatic Posture Recognition in Ambient-Assisted Living.

Human Action Recognition (HAR) is a rapidly evolving field impacting numerous domains, among which is Ambient Assisted Living (AAL). In such a context, the aim of HAR is meeting the needs of frail individuals, whether elderly and/or disabled and promoting autonomous, safe and secure living. To this goal, we propose a monitoring system detecting dangerous situations by classifying human postures through Artificial Intelligence (AI) solutions. The developed algorithm works on a set of features computed from the skeleton data provided by four Kinect One systems simultaneously recording the scene from different angles and identifying the posture of the subject in an ecological context within each recorded frame. Here, we compare the recognition abilities of Multi-Layer Perceptron (MLP) and Long-Short Term Memory (LSTM) Sequence networks. Starting from the set of previously selected features we performed a further feature selection based on an SVM algorithm for the optimization of the MLP network and used a genetic algorithm for selecting the features for the LSTM sequence model. We then optimized the architecture and hyperparameters of both models before comparing their performances. The best MLP model (3 hidden layers and a Softmax output layer) achieved 78.4%, while the best LSTM (2 bidirectional LSTM layers, 2 dropout and a fully connected layer) reached 85.7%. The analysis of the performances on individual classes highlights the better suitability of the LSTM approach.

Skeleton data pre-processing for human pose recognition using Neural Network.

Automatic monitoring of daily living activities can greatly improve the possibility of living autonomously for frail individuals. Pose recognition based on skeleton tracking data is promising for identifying dangerous situations and trigger external intervention or other alarms, while avoiding privacy issues and the need for patient compliance. Here we present the benefits of pre-processing Kinect-recorded skeleton data to limit the several errors produced by the system when the subject is not in ideal tracking conditions. The accuracy of our two hidden layers MLP classifier improved from about 82% to over 92% in recognizing actors in four different poses: standing, sitting, lying and dangerous sitting.

Automatic Pose Recognition for Monitoring Dangerous Situations in Ambient-Assisted Living.

Continuous monitoring of frail individuals for detecting dangerous situations during their daily living at home can be a powerful tool toward their inclusion in the society by allowing living independently while safely. To this goal we developed a pose recognition system tailored to disabled students living in college dorms and based on skeleton tracking through four Kinect One devices independently recording the inhabitant with different viewpoints, while preserving the individual's privacy. The system is intended to classify each data frame and provide the classification result to a further decision-making algorithm, which may trigger an alarm based on the classified pose and the location of the subject with respect to the furniture in the room. An extensive dataset was recorded on 12 individuals moving in a mockup room and undertaking four poses to be recognized: standing, sitting, lying down, and "dangerous sitting." The latter consists of the subject slumped in a chair with his/her head lying forward or backward as if unconscious. Each skeleton frame was labeled and represented using 10 discriminative features: three skeletal joint vertical coordinates and seven relative and absolute angles describing articular joint positions and body segment orientation. In order to classify the pose of the subject in each skeleton frame we built a two hidden layers multi-layer perceptron neural network with a "SoftMax" output layer, which we trained on the data from 10 of the 12 subjects (495,728 frames), with the data from the two remaining subjects representing the test set (106,802 frames). The system achieved very promising results, with an average accuracy of 83.9% (ranging 82.7 and 94.3% in each of the four classes). Our work proves the usefulness of human pose recognition based on machine learning in the field of safety monitoring in assisted living conditions.

Parahippocampal Involvement in Mesial Temporal Lobe Epilepsy with Hippocampal Sclerosis: A Proof of Concept from Memory-Guided Saccades.

OBJECTIVE: Mesial temporal lobe epilepsy with hippocampal sclerosis (MTLE-HS) may involve extrahippocampal areas of structural damage and dysfunction. The accuracy of medium-term spatial memory can be tested by memory-guided saccades (MGS) to evaluate a functional impairment of the parahippocampal cortex (PHC), while voxel-based morphometry (VBM) analysis can be used to detect a structural damage of the latter region. METHODS: MGS with 3- and 30-s memorization delays were compared between 7 patients affected by right MTLE-HS (r-MTLE-HS), 6 patients affected by left MTLE-HS, and 13 healthy controls. The same subjects underwent brain MRI for a VBM analysis. Correlation analysis was performed between the results of VBM and MGS and with patients' clinical data. RESULTS: Right MTLE-HS patients showed impaired accuracy of leftward MGS with a 30-s memorization delay; their gray-matter volume was reduced in the right hippocampus and inferior temporal gyrus, and bilaterally in the cerebellum. Left MTLE-HS patients had normal MGS accuracy; their gray-matter volume was reduced in the left hippocampus, in the right-inferior temporal gyrus and corpus callosus, and bilaterally in the insular cortex and in the cerebellum. The difference between right and left parahippocampal volumes correlated with MGS accuracy, while right and left hippocampal volumes did not. Hippocampal and parahippocampal volume did not correlate with clinical variables such as febrile seizures, age at disease onset, disease duration, and seizure frequency. CONCLUSION: MGS abnormalities suggested the functional involvement of the right PHC in patients with r-MTLE-HS, supporting a right lateralization of spatial memory control and showing a relation between functional impairment and degree of atrophy.

A new device to assess static ocular torsion.

In clinical settings, static ocular torsion is assessed by taking a fundus photograph and measuring the angle between a horizontal line and the line connecting the fovea to the head of the optic nerve (centro-cecal axis rotation; CCAR). We developed and tested a system specifically aimed at CCAR measurements, based on low-cost commercial hardware, and that implements an adaptive research algorithm that selects and presents bright dots on a monitor to outline the borders of the blind spot, locate its center, and measure CCAR. We examined 10 healthy subjects who underwent four CCAR measurements to evaluate the reliability of the system and compared our results with those of fundus photographic examination. Our data showed an excyclophoria, with mean ocular torsion of 6.4° in the right and 6.7° in the left eye. These values are in keeping with those in the literature. Moreover, the values of the intraclass correlation coefficients suggest excellent reliability of the technique.

θ-burst stimulation of the cerebellum interferes with internal representations of sensory-motor information related to eye movements in humans.

Continuous theta-burst stimulation (cTBS) applied over the cerebellum exerts long-lasting effects by modulating long-term synaptic plasticity, which is thought to be the basis of learning and behavioral adaptation. To investigate the impact of cTBS over the cerebellum on short-term sensory-motor memory, we recorded in two groups of eight healthy subject each the visually guided saccades (VGSs), the memory-guided saccades (MGSs), and the multiple memory-guided saccades (MMGSs), before and after cTBS (cTBS group) or simulated cTBS (control group). In the cTBS group, cTBS determined hypometria of contralateral centrifugal VGSs and worsened the accuracy of MMGS bilaterally. In the control group, no significant differences were found between the two recording sessions. These results indicate that cTBS over the cerebellum causes eye movement effects that last longer than the stimulus duration. The VGS contralateral hypometria suggested that we eventually inhibited the fastigial nucleus on the stimulated side. MMGSs in normal subjects have a better final accuracy with respect to MGSs. Such improvement is due to the availability in MMGSs of the efference copy of the initial reflexive saccade directed toward the same peripheral target, which provides a sensory-motor information that is memorized and then used to improve the accuracy of the subsequent volitional memory-guided saccade. Thus, we hypothesize that cTBS disrupted the capability of the cerebellum to make an internal representation of the memorized sensory-motor information to be used after a short interval for forward control of saccades.

Multiple memory-guided saccades: movement memory improves the accuracy of memory-guided saccades.

Memory-guided saccades (MGSs) with 3 s memorization delay were recorded in healthy subjects using four different paradigms: two "regular" MGS paradigms with the peripheral target lit for 0.2 s (MGS2) and for 1.8 s (MGS18); a multiple memory-guided saccade (MMGS) paradigm with the target lit for 1.8 s and the instruction to perform a visually guided saccade (VGS) towards it before the MGS; a trained memory-guided saccades (TMGSs) paradigm where the same target was presented so that the subjects should made 10 VGSs before the MGS. The longer target presentation interval (MGS18 paradigm) did not improve the accuracy of MGS. The execution of the VGSs improved the accuracy of the corrective saccades made after the first MGS to drive the eyes closer to the target, and this improvement was independent from the number of the VGSs (there was no difference between the MMGS and the TMGS paradigms). The VGSs provide a template that improves the capability of the corrective saccades to compensate for the residual position error at the end of the first saccade.

Binocular control of saccades in idiopathic Parkinson's disease.

We focused on the saccade disconjugate control in idiopathic Parkinson's disease patients. Our data showed that in IPD patients the saccade precision was differently impaired in the two eyes--namely, the disconjugate component was larger than in controls--more for the remembered than for the reflexive task.