A Roadmap Towards Standards for Neurally Controlled End Effectors.

The control and manipulation of various types of end effectors such as powered exoskeletons, prostheses, and 'neural' cursors by brain-machine interface (BMI) systems has been the target of many research projects. A seamless "plug and play" interface between any BMI and end effector is desired, wherein similar user's intent cause similar end effectors to behave identically. This report is based on the outcomes of an IEEE Standards Association Industry Connections working group on End Effectors for Brain-Machine Interfacing that convened to identify and address gaps in the existing standards for BMI-based solutions with a focus on the end-effector component. A roadmap towards standardization of end effectors for BMI systems is discussed by identifying current device standards that are applicable for end effectors. While current standards address basic electrical and mechanical safety, and to some extent, performance requirements, several gaps exist pertaining to unified terminologies, data communication protocols, patient safety and risk mitigation.

An empirical comparison of neural networks and machine learning algorithms for EEG gait decoding.

Previous studies of Brain Computer Interfaces (BCI) based on scalp electroencephalography (EEG) have demonstrated the feasibility of decoding kinematics for lower limb movements during walking. In this computational study, we investigated offline decoding analysis with different models and conditions to assess how they influence the performance and stability of the decoder. Specifically, we conducted three computational decoding experiments that investigated decoding accuracy: (1) based on delta band time-domain features, (2) when downsampling data, (3) of different frequency band features. In each experiment, eight different decoder algorithms were compared including the current state-of-the-art. Different tap sizes (sample window sizes) were also evaluated for a real-time applicability assessment. A feature of importance analysis was conducted to ascertain which features were most relevant for decoding; moreover, the stability to perturbations was assessed to quantify the robustness of the methods. Results indicated that generally the Gated Recurrent Unit (GRU) and Quasi Recurrent Neural Network (QRNN) outperformed other methods in terms of decoding accuracy and stability. Previous state-of-the-art Unscented Kalman Filter (UKF) still outperformed other decoders when using smaller tap sizes, with fast convergence in performance, but occurred at a cost to noise vulnerability. Downsampling and the inclusion of other frequency band features yielded overall improvement in performance. The results suggest that neural network-based decoders with downsampling or a wide range of frequency band features could not only improve decoder performance but also robustness with applications for stable use of BCIs.

Deep learning for electroencephalogram (EEG) classification tasks: a review.

OBJECTIVE: Electroencephalography (EEG) analysis has been an important tool in neuroscience with applications in neuroscience, neural engineering (e.g. Brain-computer interfaces, BCI's), and even commercial applications. Many of the analytical tools used in EEG studies have used machine learning to uncover relevant information for neural classification and neuroimaging. Recently, the availability of large EEG data sets and advances in machine learning have both led to the deployment of deep learning architectures, especially in the analysis of EEG signals and in understanding the information it may contain for brain functionality. The robust automatic classification of these signals is an important step towards making the use of EEG more practical in many applications and less reliant on trained professionals. Towards this goal, a systematic review of the literature on deep learning applications to EEG classification was performed to address the following critical questions: (1) Which EEG classification tasks have been explored with deep learning? (2) What input formulations have been used for training the deep networks? (3) Are there specific deep learning network structures suitable for specific types of tasks? APPROACH: A systematic literature review of EEG classification using deep learning was performed on Web of Science and PubMed databases, resulting in 90 identified studies. Those studies were analyzed based on type of task, EEG preprocessing methods, input type, and deep learning architecture. MAIN RESULTS: For EEG classification tasks, convolutional neural networks, recurrent neural networks, deep belief networks outperform stacked auto-encoders and multi-layer perceptron neural networks in classification accuracy. The tasks that used deep learning fell into five general groups: emotion recognition, motor imagery, mental workload, seizure detection, event related potential detection, and sleep scoring. For each type of task, we describe the specific input formulation, major characteristics, and end classifier recommendations found through this review. SIGNIFICANCE: This review summarizes the current practices and performance outcomes in the use of deep learning for EEG classification. Practical suggestions on the selection of many hyperparameters are provided in the hope that they will promote or guide the deployment of deep learning to EEG datasets in future research.

A mobile brain-body imaging dataset recorded during treadmill walking with a brain-computer interface.

We present a mobile brain-body imaging (MoBI) dataset acquired during treadmill walking in a brain-computer interface (BCI) task. The data were collected from eight healthy subjects, each having three identical trials. Each trial consisted of three conditions: standing, treadmill walking, and treadmill walking with a closed-loop BCI. During the BCI condition, subjects used their brain activity to control a virtual avatar on a screen to walk in real-time. Robust procedures were designed to record lower limb joint angles (bilateral hip, knee, and ankle) using goniometers synchronized with 60-channel scalp electroencephalography (EEG). Additionally, electrooculogram (EOG), EEG electrodes impedance, and digitized EEG channel locations were acquired to aid artifact removal and EEG dipole-source localization. This dataset is unique in that it is the first published MoBI dataset recorded during walking. It is useful in addressing several important open research questions, such as how EEG is coupled with gait cycle during closed-loop BCI, how BCI influences neural activity during walking, and how a BCI decoder may be optimized.

Brain-machine interfaces for controlling lower-limb powered robotic systems.

OBJECTIVE: Lower-limb, powered robotics systems such as exoskeletons and orthoses have emerged as novel robotic interventions to assist or rehabilitate people with walking disabilities. These devices are generally controlled by certain physical maneuvers, for example pressing buttons or shifting body weight. Although effective, these control schemes are not what humans naturally use. The usability and clinical relevance of these robotics systems could be further enhanced by brain-machine interfaces (BMIs). A number of preliminary studies have been published on this topic, but a systematic understanding of the experimental design, tasks, and performance of BMI-exoskeleton systems for restoration of gait is lacking. APPROACH: To address this gap, we applied standard systematic review methodology for a literature search in PubMed and EMBASE databases and identified 11 studies involving BMI-robotics systems. The devices, user population, input and output of the BMIs and robot systems respectively, neural features, decoders, denoising techniques, and system performance were reviewed and compared. MAIN RESULTS: Results showed BMIs classifying walk versus stand tasks are the most common. The results also indicate that electroencephalography (EEG) is the only recording method for humans. Performance was not clearly presented in most of the studies. Several challenges were summarized, including EEG denoising, safety, responsiveness and others. SIGNIFICANCE: We conclude that lower-body powered exoskeletons with automated gait intention detection based on BMIs open new possibilities in the assistance and rehabilitation fields, although the current performance, clinical benefits and several key challenging issues indicate that additional research and development is required to deploy these systems in the clinic and at home. Moreover, rigorous EEG denoising techniques, suitable performance metrics, consistent trial reporting, and more clinical trials are needed to advance the field.

Real-time EEG-based brain-computer interface to a virtual avatar enhances cortical involvement in human treadmill walking.

Recent advances in non-invasive brain-computer interface (BCI) technologies have shown the feasibility of neural decoding for both users' gait intent and continuous kinematics. However, the dynamics of cortical involvement in human upright walking with a closed-loop BCI has not been investigated. This study aims to investigate the changes of cortical involvement in human treadmill walking with and without BCI control of a walking avatar. Source localization revealed significant differences in cortical network activity between walking with and without closed-loop BCI control. Our results showed sustained α/µ suppression in the Posterior Parietal Cortex and Inferior Parietal Lobe, indicating increases of cortical involvement during walking with BCI control. We also observed significant increased activity of the Anterior Cingulate Cortex (ACC) in the low frequency band suggesting the presence of a cortical network involved in error monitoring and motor learning. Additionally, the presence of low γ modulations in the ACC and Superior Temporal Gyrus may associate with increases of voluntary control of human gait. This work is a further step toward the development of a novel training paradigm for improving the efficacy of rehabilitation in a top-down approach.

Multi-Trial Gait Adaptation of Healthy Individuals during Visual Kinematic Perturbations.

Optimizing rehabilitation strategies requires understanding the effects of contextual cues on adaptation learning. Prior studies have examined these effects on the specificity of split-belt walking adaptation, showing that contextual visual cues can be manipulated to modulate the magnitude, transfer, and washout of split-belt-induced learning in humans. Specifically, manipulating the availability of vision during training or testing phases of learning resulted in differences in adaptive mechanisms for temporal and spatial features of walking. However, multi-trial locomotor training has been rarely explored when using visual kinematic gait perturbations. In this study, we investigated multi-trial locomotor adaptation in ten healthy individuals while applying visual kinematic perturbations. Subjects were instructed to control a moving cursor, which represented the position of their heel, to follow a prescribed heel path profile displayed on a monitor. The perturbations were introduced by scaling all of the lower limb joint angles by a factor of 0.7 (i.e., a gain change), resulting in visual feedback errors between subjects' heel trajectories and the prescribed path profiles. Our findings suggest that, with practice, the subjects learned, albeit with different strategies, to reduce the tracking errors and showed faster response time in later trials. Moreover, the gait symmetry indices, in both the spatial and temporal domains, changed significantly during gait adaptation (P < 0.001). After-effects were present in the temporal gait symmetry index whens the visual perturbations were removed in the post-exposure period (P < 0.001), suggesting adaptation learning. These findings may have implications for developing novel gait rehabilitation interventions.

Risk management and regulations for lower limb medical exoskeletons: a review.

Gait disability is a major health care problem worldwide. Powered exoskeletons have recently emerged as devices that can enable users with gait disabilities to ambulate in an upright posture, and potentially bring other clinical benefits. In 2014, the US Food and Drug Administration approved marketing of the ReWalk™ Personal Exoskeleton as a class II medical device with special controls. Since then, Indego™ and Ekso™ have also received regulatory approval. With similar trends worldwide, this industry is likely to grow rapidly. On the other hand, the regulatory science of powered exoskeletons is still developing. The type and extent of probable risks of these devices are yet to be understood, and industry standards are yet to be developed. To address this gap, Manufacturer and User Facility Device Experience, Clinicaltrials.gov, and PubMed databases were searched for reports of adverse events and inclusion and exclusion criteria involving the use of lower limb powered exoskeletons. Current inclusion and exclusion criteria, which can determine probable risks, were found to be diverse. Reported adverse events and identified risks of current devices are also wide-ranging. In light of these findings, current regulations, standards, and regulatory procedures for medical device applications in the USA, Europe, and Japan were also compared. There is a need to raise awareness of probable risks associated with the use of powered exoskeletons and to develop adequate countermeasures, standards, and regulations for these human-machine systems. With appropriate risk mitigation strategies, adequate standards, comprehensive reporting of adverse events, and regulatory oversight, powered exoskeletons may one day allow individuals with gait disabilities to safely and independently ambulate.

Powered exoskeletons for bipedal locomotion after spinal cord injury.

OBJECTIVE: Powered exoskeletons promise to increase the quality of life of people with lower-body paralysis or weakened legs by assisting or restoring legged mobility while providing health benefits across multiple physiological systems. Here, a systematic review of the literature on powered exoskeletons addressed critical questions: What is the current evidence of clinical efficacy for lower-limb powered exoskeletons? What are the benefits and risks for individuals with spinal cord injury (SCI)? What are the levels of injury considered in such studies? What are their outcome measures? What are the opportunities for the next generation exoskeletons? APPROACH: A systematic search of online databases was performed to identify clinical trials and safety or efficacy studies with lower-limb powered exoskeletons for individuals with SCI. Twenty-two studies with eight powered exoskeletons thus selected, were analyzed based on the protocol design, subject demographics, study duration, and primary/secondary outcome measures for assessing exoskeleton's performance in SCI subjects. MAIN RESULTS: Findings show that the level of injury varies across studies, with T10 injuries being represented in 45.4% of the studies. A categorical breakdown of outcome measures revealed 63% of these measures were gait and ambulation related, followed by energy expenditure (16%), physiological improvements (13%), and usability and comfort (8%). Moreover, outcome measures varied across studies, and none had measures spanning every category, making comparisons difficult. SIGNIFICANCE: This review of the literature shows that a majority of current studies focus on thoracic level injury as well as there is an emphasis on ambulatory-related primary outcome measures. Future research should: 1) develop criteria for optimal selection and training of patients most likely to benefit from this technology, 2) design multimodal gait intention detection systems that engage and empower the user, 3) develop real-time monitoring and diagnostic capabilities, and 4) adopt comprehensive metrics for assessing safety, benefits, and usability.

Gait adaptation to visual kinematic perturbations using a real-time closed-loop brain-computer interface to a virtual reality avatar.

OBJECTIVE: The control of human bipedal locomotion is of great interest to the field of lower-body brain-computer interfaces (BCIs) for gait rehabilitation. While the feasibility of closed-loop BCI systems for the control of a lower body exoskeleton has been recently shown, multi-day closed-loop neural decoding of human gait in a BCI virtual reality (BCI-VR) environment has yet to be demonstrated. BCI-VR systems provide valuable alternatives for movement rehabilitation when wearable robots are not desirable due to medical conditions, cost, accessibility, usability, or patient preferences. APPROACH: In this study, we propose a real-time closed-loop BCI that decodes lower limb joint angles from scalp electroencephalography (EEG) during treadmill walking to control a walking avatar in a virtual environment. Fluctuations in the amplitude of slow cortical potentials of EEG in the delta band (0.1-3 Hz) were used for prediction; thus, the EEG features correspond to time-domain amplitude modulated potentials in the delta band. Virtual kinematic perturbations resulting in asymmetric walking gait patterns of the avatar were also introduced to investigate gait adaptation using the closed-loop BCI-VR system over a period of eight days. MAIN RESULTS: Our results demonstrate the feasibility of using a closed-loop BCI to learn to control a walking avatar under normal and altered visuomotor perturbations, which involved cortical adaptations. The average decoding accuracies (Pearson's r values) in real-time BCI across all subjects increased from (Hip: 0.18 ± 0.31; Knee: 0.23 ± 0.33; Ankle: 0.14 ± 0.22) on Day 1 to (Hip: 0.40 ± 0.24; Knee: 0.55 ± 0.20; Ankle: 0.29 ± 0.22) on Day 8. SIGNIFICANCE: These findings have implications for the development of a real-time closed-loop EEG-based BCI-VR system for gait rehabilitation after stroke and for understanding cortical plasticity induced by a closed-loop BCI-VR system.

Classification of finger vibrotactile input using scalp EEG.

While there are many output brain-computer interface (output BCIs) studies, few have examined the input pathway, namely decoding the sensory input. To examine the possibility of building a BCI with sensory input using scalp electroencephalography (EEG), this study builds a classifier based on Local Fisher Discriminant Analysis (LFDA) and Gaussian Mixture Model (GMM) to classify neural activity generated by vibrotactile sensory stimuli delivered to the fingers. Small vibrators were placed on the fingertips of the participant. They vibrated one by one in a random sequence while the participant sat still with eyes closed. EEG data were recorded and later used to classify which finger was vibrated. There were two tasks: one focusing on differentiating between ipsilateral fingers, the other one focusing on differentiating contralateral fingers. Decoding accuracies were high in both tasks: 97.6% and 99.3% respectively. Event-related EEG features in both amplitude and power domain are discussed.

A Closed-loop Brain Computer Interface to a Virtual Reality Avatar: Gait Adaptation to Visual Kinematic Perturbations.

The control of human bipedal locomotion is of great interest to the field of lower-body brain computer interfaces (BCIs) for rehabilitation of gait. While the feasibility of a closed-loop BCI system for the control of a lower body exoskeleton has been recently shown, multi-day closed-loop neural decoding of human gait in a virtual reality (BCI-VR) environment has yet to be demonstrated. In this study, we propose a real-time closed-loop BCI that decodes lower limb joint angles from scalp electroencephalography (EEG) during treadmill walking to control the walking movements of a virtual avatar. Moreover, virtual kinematic perturbations resulting in asymmetric walking gait patterns of the avatar were also introduced to investigate gait adaptation using the closed-loop BCI-VR system over a period of eight days. Our results demonstrate the feasibility of using a closed-loop BCI to learn to control a walking avatar under normal and altered visuomotor perturbations, which involved cortical adaptations. These findings have implications for the development of BCI-VR systems for gait rehabilitation after stroke and for understanding cortical plasticity induced by a closed-loop BCI system.

An integrated neuro-robotic interface for stroke rehabilitation using the NASA X1 powered lower limb exoskeleton.

Stroke remains a leading cause of disability, limiting independent ambulation in survivors, and consequently affecting quality of life (QOL). Recent technological advances in neural interfacing with robotic rehabilitation devices are promising in the context of gait rehabilitation. Here, the X1, NASA's powered robotic lower limb exoskeleton, is introduced as a potential diagnostic, assistive, and therapeutic tool for stroke rehabilitation. Additionally, the feasibility of decoding lower limb joint kinematics and kinetics during walking with the X1 from scalp electroencephalographic (EEG) signals--the first step towards the development of a brain-machine interface (BMI) system to the X1 exoskeleton--is demonstrated.