A New Socially Assistive Robot with Integrated Serious Games for Therapies with Children with Autism Spectrum Disorder and Down Syndrome: A Pilot Study.

This work introduces a new socially assistive robot termed MARIA T21 (meaning "Mobile Autonomous Robot for Interaction with Autistics", with the addition of the acronym T21, meaning "Trisomy 21", which is used to designate individuals with Down syndrome). This new robot is used in psychomotor therapies for children with Down syndrome (contributing to improve their proprioception, postural balance, and gait) as well as in psychosocial and cognitive therapies for children with autism spectrum disorder. The robot uses, as a novelty, an embedded mini-video projector able to project Serious Games on the floor or tables to make already-established therapies funnier to these children, thus creating a motivating and facilitating effect for both children and therapists. The Serious Games were developed in Python through the library Pygame, considering theoretical bases of behavioral psychology for these children, which are integrated into the robot through the robot operating system (ROS). Encouraging results from the child-robot interaction are shown, according to outcomes obtained from the application of the Goal Attainment Scale. Regarding the Serious Games, they were considered suitable based on both the "Guidelines for Game Design of Serious Games for Children" and the "Evaluation of the Psychological Bases" used during the games' development. Thus, this pilot study seeks to demonstrate that the use of a robot as a therapeutic tool together with the concept of Serious Games is an innovative and promising tool to help health professionals in conducting therapies with children with autistic spectrum disorder and Down syndrome. Due to health issues imposed by the COVID-19 pandemic, the sample of children was limited to eight children (one child with typical development, one with Trisomy 21, both female, and six children with ASD, one girl and five boys), from 4 to 9 years of age. For the non-typically developing children, the inclusion criterion was the existence of a conclusive diagnosis and fulfillment of at least 1 year of therapy. The protocol was carried out in an infant psychotherapy room with three video cameras, supervised by a group of researchers and a therapist. The experiments were separated into four steps: The first stage was composed of a robot introduction followed by an approximation between robot and child to establish eye contact and assess proxemics and interaction between child/robot. In the second stage, the robot projected Serious Games on the floor, and emitted verbal commands, seeking to evaluate the child's susceptibility to perform the proposed tasks. In the third stage, the games were performed for a certain time, with the robot sending messages of positive reinforcement to encourage the child to accomplish the game. Finally, in the fourth stage, the robot finished the games and said goodbye to the child, using messages aiming to build a closer relationship with the child.

Shoulder Flexion Pre-Movement Recognition Through Subject-Specific Brain Regions to Command an Upper Limb Exoskeleton.

This work presents two brain-computer interfaces (BCIs) for shoulder pre-movement recognition using: 1) manual strategy for Electroencephalography (EEG) channels selection, and 2) subject-specific channels selection by applying non-negative factorization matrix (NMF). Besides, the proposed BCIs compute spatial features extracted from filtered EEG signals through Riemannian covariance matrices and a linear discriminant analysis (LDA) to discriminate both shoulder pre-movement and rest states. We studied on twenty-one healthy subjects different frequency ranges looking the best frequency band for shoulder pre-movement recognition. As a result, our BCI located automatically EEG channels on the contralateral moved limb, and enhancing the pre-movement recognition (ACC = 71.39 ± 12.68%, κ = 0.43 ± 0.25%). The ability of the proposed BCIs to select specific EEG locations more cortically related to the moved limb could benefit the neuro-rehabilitation process.

Assistive locomotion device with haptic feedback for guiding visually impaired people.

Robotic assistive devices are able to enhance physical stability and balance. Smart walkers, in particular, are also capable of offering cognitive support for individuals whom conventional walkers are unsuitable. However, visually impaired individuals often need additional sensorial assistance from those devices. This work proposes a smart walker with an admittance controller for guiding visually impaired individuals along a desired path. The controller uses as inputs the physical interaction between the user and the walker to provide haptic feedback hinting the path to be followed. Such controller is validated in a set of experiments with healthy individuals. At first, users were blindfolded during navigation to assess the capacity of the smart walker in providing guidance without visual input. Then, the blindfold is removed and the focus is on evaluating the human-robot interaction when the user had visual information during navigation. The results indicate that the admittance controller design and the design of the guidance path were factors impacting on the level of comfort reported by users. In addition, when the user was blindfolded, the linear velocity assumed lower values than when did not wear it, from a mean value of 0.19 m/s to 0.21 m/s.

Assessment of an Assistive Control Approach Applied in an Active Knee Orthosis Plus Walker for Post-Stroke Gait Rehabilitation.

The goal of this study is the assessment of an assistive control approach applied to an active knee orthosis plus a walker for gait rehabilitation. The study evaluates post-stroke patients and healthy subjects (control group) in terms of kinematics, kinetics, and muscle activity. Muscle and gait information of interest were acquired from their lower limbs and trunk, and a comparison was conducted between patients and control group. Signals from plantar pressure, gait phase, and knee angle and torque were acquired during gait, which allowed us to verify that the stance control strategy proposed here was efficient at improving the patients' gaits (comparing their results to the control group), without the necessity of imposing a fixed knee trajectory. An innovative evaluation of trunk muscles related to the maintenance of dynamic postural equilibrium during gait assisted by our active knee orthosis plus walker was also conducted through inertial sensors. An increase in gait cycle (stance phase) was also observed when comparing the results of this study to our previous work. Regarding the kinematics, the maximum knee torque was lower for patients when compared to the control group, which implies that our orthosis did not demand from the patients a knee torque greater than that for healthy subjects. Through surface electromyography (sEMG) analysis, a significant reduction in trunk muscle activation and fatigability, before and during the use of our orthosis by patients, was also observed. This suggest that our orthosis, together with the assistive control approach proposed here, is promising and could be considered to complement post-stroke patient gait rehabilitation.

Subject-specific EEG channel selection using non-negative matrix factorization for lower-limb motor imagery recognition.

OBJECTIVE: This study aims to propose and validate a subject-specific approach to recognize two different cognitive neural states (relax and pedaling motor imagery (MI)) by selecting the relevant electroencephalogram (EEG) channels. The main aims of the proposed work are: (i) to reduce the computational complexity of the BCI systems during MI detection by selecting the relevant EEG channels, (ii) to reduce the amount of data overfitting that may arise due to unnecessary channels and redundant features, and (iii) to reduce the classification time for real-time BCI applications. APPROACH: The proposed method selects subject-specific EEG channels and features based on their MI. In this work, we make use of non-negative matrix factorization to extract the weight of the EEG channels based on their contribution to MI detection. Further, the neighborhood component analysis is used for subject-specific feature selection. MAIN RESULTS: We executed the experiments using EEG signals recorded for MI where ten healthy subjects performed MI movement of the lower limb to generate motor commands. An average accuracy of 96.66%, average true positive rate (TPR) of 97.77%, average false positives rate of 4.44%, and average Kappa of 93.33% were obtained. The proposed subject-specific EEG channel selection based MI recognition system provides 13.20% improvement in detection accuracy, and 27% improvement in Kappa value with less number of EEG channels compared to the results obtained using all EEG channels. SIGNIFICANCE: The proposed subject-specific BCI system has been found significantly advantageous compared to the typical approach of using a fixed channel configuration. This work shows that fewer EEG channels not only reduce computational complexity and processing time (two times faster) but also improve the MI detection performance. The proposed method selects EEG locations related to the foot movement, which may be relevant for neuro-rehabilitation using lower-limb movements that may provide a real-time and more natural interface between patient and robotic device.

Emotion analysis in children through facial emissivity of infrared thermal imaging.

Physiological signals may be used as objective markers to identify emotions, which play relevant roles in social and daily life. To measure these signals, the use of contact-free techniques, such as Infrared Thermal Imaging (IRTI), is indispensable to individuals who have sensory sensitivity. The goal of this study is to propose an experimental design to analyze five emotions (disgust, fear, happiness, sadness and surprise) from facial thermal images of typically developing (TD) children aged 7-11 years using emissivity variation, as recorded by IRTI. For the emotion analysis, a dataset considered emotional dimensions (valence and arousal), facial bilateral sides and emotion classification accuracy. The results evidence the efficiency of the experimental design with interesting findings, such as the correlation between the valence and the thermal decrement in nose; disgust and happiness as potent triggers of facial emissivity variations; and significant emissivity variations in nose, cheeks and periorbital regions associated with different emotions. Moreover, facial thermal asymmetry was revealed with a distinct thermal tendency in the cheeks, and classification accuracy reached a mean value greater than 85%. From the results, the emissivity variations were an efficient marker to analyze emotions in facial thermal images, and IRTI was confirmed to be an outstanding technique to study emotions. This study contributes a robust dataset to analyze the emotions of 7-11-year-old TD children, an age range for which there is a gap in the literature.

Motor Imagery Classification with Covariance Matrices and Non-Negative Matrix Factorization.

In this paper, we aim at finding the smallest set of EEG channels that can ensure highly accurate classification of motor imagery (MI) dataset and maintain the optimum Kappa score. Non-negative matrix factorization (NMF) is used for important and discriminant EEG channel selection. Further, the theory of Riemannian geometry in the manifold of covariance matrices is used for feature extraction. At last, the neighborhood component feature selection (NCFS) algorithm is used to select the small subset of important features from the given set of features. The significance of the proposed work is two-fold: 1) it greatly reduces the time complexity and the amount of overfitting by reducing the unnecessary EEG channels and redundant features. 2) it increases the classification accuracy of the model by selecting only subject-specific EEG channels. The proposed algorithm is tested on BCI Competition IV,2a dataset to validate the performance. The proposed approach has achieved 77.91% average classification accuracy and 0.626 mean Kappa score.

Control of a robotic knee exoskeleton for assistance and rehabilitation based on motion intention from sEMG

Introduction: This work presents the development of a novel robotic knee exoskeleton controlled by motion intention based on sEMG, which uses admittance control to assist people with reduced mobility and improve their locomotion. Clinical research remark that these devices working in constant interaction with the neuromuscular and skeletal human system improves functional compensation and rehabilitation. Hence, the users become an active part of the training/rehabilitation, facilitating their involvement and improving their neural plasticity. For recognition of the lower-limb motion intention and discrimination of knee movements, sEMG from both lower-limb and trunk are used, which implies a new approach to control robotic assistive devices. Methods A control system that includes a stage for human-motion intention recognition (HMIR), based on techniques to classify motion classes related to knee joint were developed. For translation of the user's intention to a desired state for the robotic knee exoskeleton, the system also includes a finite state machine and admittance, velocity and trajectory controllers with a function that allows stopping the movement according to the users intention. Results The proposed HMIR showed an accuracy between 76% to 83% for lower-limb muscles, and 71% to 77% for trunk muscles to classify motor classes of lower-limb movements. Experimental results of the controller showed that the admittance controller proposed here offers knee support in 50% of the gait cycle and assists correctly the motion classes. Conclusion The robotic knee exoskeleton introduced here is an alternative method to empower knee movements using sEMG signals from lower-limb and trunk muscles.

Knee Impedance Modulation to Control an Active Orthosis Using Insole Sensors.

Robotic devices for rehabilitation and gait assistance have greatly advanced with the objective of improving both the mobility and quality of life of people with motion impairments. To encourage active participation of the user, the use of admittance control strategy is one of the most appropriate approaches, which requires methods for online adjustment of impedance components. Such approach is cited by the literature as a challenge to guaranteeing a suitable dynamic performance. This work proposes a method for online knee impedance modulation, which generates variable gains through the gait cycle according to the users' anthropometric data and gait sub-phases recognized with footswitch signals. This approach was evaluated in an active knee orthosis with three variable gain patterns to obtain a suitable condition to implement a stance controller: two different gain patterns to support the knee in stance phase, and a third pattern for gait without knee support. The knee angle and torque were measured during the experimental protocol to compare both temporospatial parameters and kinematics data with other studies of gait with knee exoskeletons. The users rated scores related to their satisfaction with both the device and controller through QUEST questionnaires. Experimental results showed that the admittance controller proposed here offered knee support in 50% of the gait cycle, and the walking speed was not significantly different between the three gain patterns (p = 0.067). A positive effect of the controller on users regarding safety during gait was found with a score of 4 in a scale of 5. Therefore, the approach demonstrates good performance to adjust impedance components providing knee support in stance phase.

Adaptation of a smart walker for stroke individuals: a study on sEMG and accelerometer signals

Abstract Introduction: Stroke is a leading cause of neuromuscular system damages, and researchers have been studying and developing robotic devices to assist affected people. Depending on the damage extension, the gait of these people can be impaired, making devices, such as smart walkers, useful for rehabilitation. The goal of this work is to analyze changes in muscle patterns on the paretic limb during free and walker-assisted gaits in stroke individuals, through accelerometry and surface electromyography (sEMG). Methods The analyzed muscles were vastus medialis, biceps femoris, tibialis anterior and gastrocnemius medialis. The volunteers walked three times on a straight path in free gait and, further, three times again, but now using the smart walker, to help them with the movements. Then, the data from gait pattern and muscle signals collected by sEMG and accelerometers were analyzed and statistical analyses were applied. Results The accelerometry allowed gait phase identification (stance and swing), and sEMG provided information about muscle pattern variations, which were detected in vastus medialis (onset and offset; p = 0.022) and biceps femoris (offset; p = 0.025). Additionally, comparisons between free and walker-assisted gaits showed significant reduction in speed (from 0.45 to 0.30 m/s; p = 0.021) and longer stance phase (from 54.75 to 60.34%; p = 0.008). Conclusions Variations in muscle patterns were detected in vastus medialis and biceps femoris during the experiments, besides user speed reduction and longer stance phase when the walker-assisted gait is compared with the free gait.

Dexterous hand gestures recognition based on low-density sEMG signals for upper-limb forearm amputees

Abstract Introduction Intuitive prosthesis control is one of the most important challenges in order to reduce the user effort in learning how to use an artificial hand. This work presents the development of a novel method for pattern recognition of sEMG signals able to discriminate, in a very accurate way, dexterous hand and fingers movements using a reduced number of electrodes, which implies more confidence and usability for amputees. Methods The system was evaluated for ten forearm amputees and the results were compared with the performance of able-bodied subjects. Multiple sEMG features based on fractal analysis (detrended fluctuation analysis and Higuchi's fractal dimension) combined with traditional magnitude-based features were analyzed. Genetic algorithms and sequential forward selection were used to select the best set of features. Support vector machine (SVM), K-nearest neighbors (KNN) and linear discriminant analysis (LDA) were analyzed to classify individual finger flexion, hand gestures and different grasps using four electrodes, performing contractions in a natural way to accomplish these tasks. Statistical significance was computed for all the methods using different set of features, for both groups of subjects (able-bodied and amputees). Results The results showed average accuracy up to 99.2% for able-bodied subjects and 98.94% for amputees using SVM, followed very closely by KNN. However, KNN also produces a good performance, as it has a lower computational complexity, which implies an advantage for real-time applications. Conclusion The results show that the method proposed is promising for accurately controlling dexterous prosthetic hands, providing more functionality and better acceptance for amputees.

Comparison of the influence of stimuli color on Steady-State Visual Evoked Potentials

IntroductionThe main idea of a traditional Steady State Visually Evoked Potentials (SSVEP)-BCI is the activation of commands through gaze control. For this purpose, the retina of the eye is excited by a stimulus at a certain frequency. Several studies have shown effects related to different kind of stimuli, frequencies, window lengths, techniques of feature extraction and even classification. So far, none of the previous studies has performed a comparison of performance of stimuli colors through LED technology. This study addresses precisely this important aspect and would be a great contribution to the topic of SSVEP-BCIs. Additionally, the performance of different colors at different frequencies and the visual comfort were evaluated in each case.MethodsLEDs of four different colors (red, green, blue and yellow) flickering at four distinct frequencies (8, 11, 13 and 15 Hz) were used. Twenty subjects were distributed in two groups performing different protocols. Multivariate Synchronization Index (MSI) was the technique adopted as feature extractor.ResultsThe accuracy was gradually enhanced with the increase of the time window. From our observations, the red color provides, in most frequencies, both highest rates of accuracy and Information Transfer Rate (ITR) for detection of SSVEP.ConclusionAlthough the red color has presented higher ITR, this color was turned in the less comfortable one and can even elicit epileptic responses according to the literature. For this reason, the green color is suggested as the best choice according to the proposed rules. In addition, this color has shown to be safe and accurate for an SSVEP-BCI.

A new approach for SSVEP detection using PARAFAC and canonical correlation analysis.

This paper presents a new way for automatic detection of SSVEPs through correlation analysis between tensor models. 3-way EEG tensor of channel × frequency × time is decomposed into constituting factor matrices using PARAFAC model. PARAFAC analysis of EEG tensor enables us to decompose multichannel EEG into constituting temporal, spectral and spatial signatures. SSVEPs characterized with localized spectral and spatial signatures are then detected exploiting a correlation analysis between extracted signatures of the EEG tensor and the corresponding simulated signatures of all target SSVEP signals. The SSVEP that has the highest correlation is selected as the intended target. Two flickers blinking at 8 and 13 Hz were used as visual stimuli and the detection was performed based on data packets of 1 second without overlapping. Five subjects participated in the experiments and the highest classification rate of 83.34% was achieved, leading to the Information Transfer Rate (ITR) of 21.01 bits/min.

Towards an architecture of a hybrid BCI based on SSVEP-BCI and passive-BCI.

Recent decades have seen BCI applications as a novel and promising new channel of communication, control and entertainment for disabled and healthy people. However, BCI technology can be prone to errors due to the basic emotional state of the user: the performance of reactive and active BCIs decrease when user becomes stressed or bored, for example. Passive-BCI is a recent approach that fuses BCI technology with cognitive monitoring, providing valuable information about the user's intentions, the situational interpretations and mainly the emotional state. In this work, an architecture composed by passive-BCI co-working with SSVEP-BCI is proposed, with the aim of improving the performance of the reactive-BCI. The possibility of adjusting recognition characteristics of SSVEP-BCIs using a passive-BCI output is evaluated. In this sense, two ways to recover the accuracy of SSVEP are presented in this paper: 1) Adjusting of Amplitude of the SSVEP and 2) Adjusting of Frequency of the SSVEP response. The results are promising, because accuracy of SSVEP-BCI can be recovered in the case that it was reduced by the BCI user's emotional state.

Adaptive BCI based on software agents.

The selection of features is generally the most difficult field to model in BCIs. Therefore, time and effort are invested in individual feature selection prior to data set training. Another great difficulty regarding the model of the BCI topology is the brain signal variability between users. How should this topology be in order to implement a system that can be used by large number of users with an optimal set of features? The proposal presented in this paper allows for obtaining feature reduction and classifier selection based on software agents. The software agents contain Genetic Algorithms (GA) and a cost function. GA used entropy and mutual information to choose the number of features. For the classifier selection a cost function was defined. Success rate and Cohen's Kappa coefficient are used as parameters to evaluate the classifiers performance. The obtained results allow finding a topology represented as a neural model for an adaptive BCI, where the number of the channels, features and the classifier are interrelated. The minimal subset of features and the optimal classifier were obtained with the adaptive BCI. Only three EEG channels were needed to obtain a success rate of 93% for the BCI competition III data set IVa.

Robust human machine interface based on head movements applied to assistive robotics.

This paper presents an interface that uses two different sensing techniques and combines both results through a fusion process to obtain the minimum-variance estimator of the orientation of the user's head. Sensing techniques of the interface are based on an inertial sensor and artificial vision. The orientation of the user's head is used to steer the navigation of a robotic wheelchair. Also, a control algorithm for assistive technology system is presented. The system is evaluated by four individuals with severe motors disability and a quantitative index was developed, in order to objectively evaluate the performance. The results obtained are promising since most users could perform the proposed tasks with the robotic wheelchair.

Fractal based complexity measure and variation in force during sustained isometric muscle contraction: effect of aging.

This study has investigated the effect of age on the fractal based complexity measure of muscle activity and variance in the force of isometric muscle contraction. Surface electromyogram (sEMG) and force of muscle contraction were recorded from 40 healthy subjects categorized into: Group 1: Young - age range 20-30; 10 Males and 10 Females, Group 2: Old - age range 55-70; 10 Males and 10 Females during isometric exercise at Maximum Voluntary contraction (MVC). The results show that there is a reduction in the complexity of surface electromyogram (sEMG) associated with aging. The results demonstrate that there is an increase in the coefficient of variance (CoV) of the force of muscle contraction and a decrease in complexity of sEMG for the Old age group when compared with the Young age group.

Comparative muscle study fatigue with sEMG signals during the isotonic and isometric tasks for diagnostics purposes.

The study of fatigue is an important tool for diagnostics of disease, sports, ergonomics and robotics areas. This work deals with the analysis of sEMG most important fatigue muscle indicators with use of signal processing in isometric and isotonic tasks with the propose of standardizing fatigue protocol to select the data acquisition and processing with diagnostic proposes. As a result, the slope of the RMS, ARV and MNF indicators were successful to describe the fatigue behavior expected. Whereas that, MDF and AIF indicators failed in the description of fatigue. Similarly, the use of a constant load for sEMG data acquisition was the best strategy in both tasks.

Effect of age and gender on the surface electromyogram during various levels of isometric contraction.

This study reports the effects of age and gender on the surface electromyogram while performing isometric contraction. Experiments were conducted with two age groups--Young (Age: 20-29) and Old (Age: 60-69) where they performed sustained isometric contractions at various force levels (50%, 75%, 100% of maximum voluntary contraction). Traditional features such as root mean square (RMS) and median frequency (MDF) were computed from the recorded sEMG. The result indicates that the MDF of sEMG was not significantly affected by age, but was impacted by gender in both age groups. Also there was a significant change in the RMS of sEMG with age and gender at all levels of contraction. The results also indicate a large inter-subject variation. This study will provide an understanding of the underlying physiological effects of muscle contraction and muscle fatigue in different cohorts.

Robotic wheelchair commanded by SSVEP, motor imagery and word generation.

This work presents a robotic wheelchair that can be commanded by a Brain Computer Interface (BCI) through Steady-State Visual Evoked Potential (SSVEP), Motor Imagery and Word Generation. When using SSVEP, a statistical test is used to extract the evoked response and a decision tree is used to discriminate the stimulus frequency, allowing volunteers to online operate the BCI, with hit rates varying from 60% to 100%, and guide a robotic wheelchair through an indoor environment. When using motor imagery and word generation, three mental task are used: imagination of left or right hand, and imagination of generation of words starting with the same random letter. Linear Discriminant Analysis is used to recognize the mental tasks, and the feature extraction uses Power Spectral Density. The choice of EEG channel and frequency uses the Kullback-Leibler symmetric divergence and a reclassification model is proposed to stabilize the classifier.