End-to-End Estimation of Hand- and Wrist Forces From Raw Intramuscular EMG Signals Using LSTM Networks.

Processing myoelectrical activity in the forearm has for long been considered a promising framework to allow transradial amputees to control motorized prostheses. In spite of expectations, contemporary muscle-computer interfaces built for this purpose typically fail to satisfy one or more important desiderata, such as accuracy, robustness, and/or naturalness of control, in part due to difficulties in acquiring high-quality signals continuously outside laboratory conditions. In light of such problems, surgically implanted electrodes have been made a viable option that allows for long-term acquisition of intramuscular electromyography (iEMG) measurements of spatially precise origin. As it stands, the question of how information embedded in such signals is best extracted and combined across multiple channels remains open. This study presents and evaluates an approach to this end that uses deep neural networks based on the Long Short-Term Memory (LSTMs) architecture to regress forces exerted by multiple degrees of freedom (DoFs) from multichannel iEMG. Three deep learning models, representing three distinct regression strategies, were evaluated: (I) One-to-One, wherein each DoF is separately estimated by an LSTM model processing a single iEMG channel, (II) All-to-One, wherein each DoF is separately estimated by an LSTM model processing all iEMG channels, and (III) All-to-All, wherein a single LSTM model with access to all iEMG channels estimates all DoFs simultaneously. All models operate on raw iEMG, with no preliminary feature extraction required. When evaluated on a dataset comprising six iEMG channels with concurrent force measurements acquired from 14 subjects, all LSTM strategies were found to significantly outperform a baseline feature-based linear control regression method. This finding indicates that recurrent neural networks can learn to transform raw forearm iEMG signals directly into representations that correlate with forces exerted at the level of the hand to a greater degree than simple features do. Furthermore, the All-to-All and All-to-One strategies were found to exhibit better performance than the One-to-One strategy. This finding suggests that, in spite of the spatially local nature of signals, iEMG from muscles not directly actuating the relevant DoF can provide contextual information that aid in decoding motor intent.

Can Deep Synthesis of EMG Overcome the Geometric Growth of Training Data Required to Recognize Multiarticulate Motions?

By being predicated on supervised machine learning, pattern recognition approaches to myoelectric prosthesis control require electromyography (EMG) training data collected concurrently with every detectable motion. Within this framework, calibration protocols for simultaneous control of multifunctional prosthetic hands rapidly become prohibitively long-the number of unique motions grows geometrically with the number of controllable degrees of freedom (DoFs). This paper proposes a technique intended to circumvent this combinatorial explosion. Using EMG windows from 1-DoF motions as input and EMG windows from 2-DoF motions as targets, we train generative deep learning models to synthesize EMG windows appertaining to multi-DoF motions. Once trained, such models can be used to complete datasets consisting of only 1-DoF motions, enabling simple calibration protocols with durations that scale linearly with the number of DoFs. We evaluated synthetic EMG produced in this way via a classification task using a database of forearm surface EMG collected during 1-DoF and 2-DoF motions. Multi-output classifiers were trained on either (I) real data from 1-DoF and 2-DoF motions, (II) real data from only 1-DoF motions, or (III) real data from 1-DoF motions appended with synthetic EMG from 2-DoF motions. When tested on data containing all possible motions, classifiers trained on synthetic-appended data (III) significantly outperformed classifiers trained on 1-DoF real data (II), although significantly underperformed classifiers trained on both 1- and 2-DoF real data (I) (I < 0.05). These findings suggest that it is feasible to model EMG concurrent with multiarticulate motions as nonlinear combinations of EMG from constituent 1-DoF motions, and that such modelling can be harnessed to synthesize realistic training data.

Learning regularized representations of categorically labelled surface EMG enables simultaneous and proportional myoelectric control.

BACKGROUND: Processing the surface electromyogram (sEMG) to decode movement intent is a promising approach for natural control of upper extremity prostheses. To this end, this paper introduces and evaluates a new framework which allows for simultaneous and proportional myoelectric control over multiple degrees of freedom (DoFs) in real-time. The framework uses multitask neural networks and domain-informed regularization in order to automatically find nonlinear mappings from the forearm sEMG envelope to multivariate and continuous encodings of concurrent hand- and wrist kinematics, despite only requiring categorical movement instruction stimuli signals for calibration. METHODS: Forearm sEMG with 8 channels was collected from healthy human subjects (N = 20) and used to calibrate two myoelectric control interfaces, each with two output DoFs. The interfaces were built from (I) the proposed framework, termed Myoelectric Representation Learning (MRL), and, to allow for comparisons, from (II) a standard pattern recognition framework based on Linear Discriminant Analysis (LDA). The online performances of both interfaces were assessed with a Fitts's law type test generating 5 quantitative performance metrics. The temporal stabilities of the interfaces were evaluated by conducting identical tests without recalibration 7 days after the initial experiment session. RESULTS: Metric-wise two-way repeated measures ANOVA with factors method (MRL vs LDA) and session (day 1 vs day 7) revealed a significant ([Formula: see text]) advantage for MRL over LDA in 5 out of 5 performance metrics, with metric-wise effect sizes (Cohen's [Formula: see text]) separating MRL from LDA ranging from [Formula: see text] to [Formula: see text]. No significant effect on any metric was detected for neither session nor interaction between method and session, indicating that none of the methods deteriorated significantly in control efficacy during one week of intermission. CONCLUSIONS: The results suggest that MRL is able to successfully generate stable mappings from EMG to kinematics, thereby enabling myoelectric control with real-time performance superior to that of the current commercial standard for pattern recognition (as represented by LDA). It is thus postulated that the presented MRL approach can be of practical utility for muscle-computer interfaces.

Automatic discovery of resource-restricted Convolutional Neural Network topologies for myoelectric pattern recognition.

Convolutional Neural Networks (CNNs) have been subject to extensive attention in the pattern recognition literature due to unprecedented performance in tasks of information extraction from unstructured data. Whereas available methods for supervised training of a CNN with a given network topology are well-defined with rigorous theoretical justification, procedures for the initial selection of topology are currently not. Work incorporating selection of the CNN topology has instead substantially been guided by the domain-specific expertise of the creator(s), followed by iterative improvement via empirical evaluation. This limitation of methodology is restricting in the pursuit of naturally controlled muscle-computer interfaces, where CNNs have been identified as a promising research avenue but effective topology selection heuristics are lacking. With the goal of mitigating ambiguities in topology selection, this paper presents a systematic approach wherein we apply a novel evolutionary algorithm to search a space of candidate topologies. Furthermore, we constrain the search-space by excluding topologies with excessive inference-time computational complexity, making the obtained results implementable in embedded systems. In contrast to manual topology design, our algorithm requires the user to only specify a relatively small set of intuitive hyperparameters. To validate our approach, we use it in order to create topologies for myoelectric pattern recognition via movement decoding of surface electromyography signals. By collating offline classification accuracies obtained from experiments on a collection of publicly available databases, we demonstrate that our method generates computationally lightweight topologies with performance comparable to those of available alternatives.

Extraction of Multi-Labelled Movement Information from the Raw HD-sEMG Image with Time-Domain Depth.

In contemporary muscle-computer interfaces for upper limb prosthetics there is often a trade-off between control robustness and range of executable movements. As a very low movement error rate is necessary in practical applications, this often results in a quite severe limitation of controllability; a problem growing ever more salient as the mechanical sophistication of multifunctional myoelectric prostheses continues to improve. A possible remedy for this could come from the use of multi-label machine learning methods, where complex movements can be expressed as the superposition of several simpler movements. Here, we investigate this claim by applying a multi-labeled classification scheme in the form of a deep convolutional neural network (CNN) to high density surface electromyography (HD-sEMG) recordings. We use 16 independent labels to model the movements of the hand and forearm state, representing its major degrees of freedom. By training the neural network on 16 × 8 sEMG image sequences 24 samples long with a sampling rate of 2048 Hz to detect these labels, we achieved a mean exact match rate of 78.7% and a mean Hamming loss of 2.9% across 14 healthy test subjects. With this, we demonstrate the feasibility of highly versatile and responsive sEMG control interfaces without loss of accuracy.