DistaNet: grasp-specific distance biofeedback promotes the retention of myoelectric skills.

Objective.An active myoelectric interface responds to the user's muscle signals to enable movements. Machine learning can decode user intentions from myoelectric signals. However, machine learning-based interface control lacks continuous, intuitive feedback about task performance, needed to facilitate the acquisition and retention of myoelectric control skills.Approach.We propose DistaNet as a neural network-based framework that extracts smooth, continuous, and low-dimensional signatures of the hand grasps from multi-channel myoelectric signals and provides grasp-specific biofeedback to the users.Main results.Experimental results show its effectiveness in decoding user gestures and providing biofeedback, helping users retain the acquired motor skills.Significance.We demonstrates myoelectric skill retention in a pattern recognition setting for the first time.

One-shot random forest model calibration for hand gesture decoding.

Objective.Most existing machine learning models for myoelectric control require a large amount of data to learn user-specific characteristics of the electromyographic (EMG) signals, which is burdensome. Our objective is to develop an approach to enable the calibration of a pre-trained model with minimal data from a new myoelectric user.Approach.We trained a random forest (RF) model with EMG data from 20 people collected during the performance of multiple hand grips. To adapt the decision rules for a new user, first, the branches of the pre-trained decision trees were pruned using the validation data from the new user. Then new decision trees trained merely with data from the new user were appended to the pruned pre-trained model.Results.Real-time myoelectric experiments with 18 participants over two days demonstrated the improved accuracy of the proposed approach when compared to benchmark user-specific RF and the linear discriminant analysis models. Furthermore, the RF model that was calibrated on day one for a new participant yielded significantly higher accuracy on day two, when compared to the benchmark approaches, which reflects the robustness of the proposed approach.Significance.The proposed model calibration procedure is completely source-free, that is, once the base model is pre-trained, no access to the source data from the original 20 people is required. Our work promotes the use of efficient, explainable, and simple models for myoelectric control.

Reducing Motor Variability Enhances Myoelectric Control Robustness Across Untrained Limb Positions.

The limb position effect is a multi-faceted problem, associated with decreased upper-limb prosthesis control acuity following a change in arm position. Factors contributing to this problem can arise from distinct environmental or physiological sources. Despite their differences in origin, the effect of each factor manifests similarly as increased input data variability. This variability can cause incorrect decoding of user intent. Previous research has attempted to address this by better capturing input data variability with data abundance. In this paper, we take an alternative approach and investigate the effect of reducing trial-to-trial variability by improving the consistency of muscle activity through user training. Ten participants underwent 4 days of myoelectric training with either concurrent or delayed feedback in a single arm position. At the end of training participants experienced a zero-feedback retention test in multiple limb positions. In doing so, we tested how well the skill learned in a single limb position generalized to untrained positions. We found that delayed feedback training led to more consistent muscle activity across both the trained and untrained limb positions. Analysis of patterns of activations in the delayed feedback group suggest a structured change in muscle activity occurs across arm positions. Our results demonstrate that myoelectric user-training can lead to the retention of motor skills that bring about more robust decoding across untrained limb positions. This work highlights the importance of reducing motor variability with practice, prior to examining the underlying structure of muscle changes associated with limb position.

An Attention-based Bidirectional LSTM Model for Continuous Cross-Subject Estimation of Knee Joint Angle during Running from sEMG Signals.

Accurate and robust estimation of joint kinematics via surface electromyogram (sEMG) signals provides a human-machine interaction (HMI)-based method that can be used to adequately control rehabilitation robots while performing complex movements, such as running, for motor function restoration in affected individuals. To this end, this paper proposes a deep learning-based model (AM-BiLSTM) that integrates a bidirectional long short-term memory (BiLSTM) network and an attention mechanism (AM) for robust estimation of joint kinematics. The proposed model was appraised using knee joint kinematic and sEMG signals collected from fourteen subjects who performed running at the speed of 2 m/s. The proposed model's generalizability was tested for both within- and cross-subject scenarios and compared with long short-term memory (LSTM) and multi-layer perceptron (MLP) networks in terms of normalized root-mean-square error and correlation coefficient metrics. Based on the statistical tests, the proposed AM-BiLSTM model significantly outperformed the LSTM and MLP methods in both within- and cross-subject scenarios (p<0.05) and achieved state-of-the-art performance.Clinical Relevance- The promising results of this study suggest that the AM-BiLSTM model has the potential for continuous cross-subject estimation of lower limb kinematics during running, which can be used to control sEMG-driven exoskeleton robots oriented towards rehabilitation training.

Alignment of magnetic sensing and clinical magnetomyography.

Neuromuscular diseases are a prevalent cause of prolonged and severe suffering for patients, and with the global population aging, it is increasingly becoming a pressing concern. To assess muscle activity in NMDs, clinicians and researchers typically use electromyography (EMG), which can be either non-invasive using surface EMG, or invasive through needle EMG. Surface EMG signals have a low spatial resolution, and while the needle EMG provides a higher resolution, it can be painful for the patients, with an additional risk of infection. The pain associated with the needle EMG can pose a risk for certain patient groups, such as children. For example, children with spinal muscular atrophy (type of NMD) require regular monitoring of treatment efficacy through needle EMG; however, due to the pain caused by the procedure, clinicians often rely on a clinical assessment rather than needle EMG. Magnetomyography (MMG), the magnetic counterpart of the EMG, measures muscle activity non-invasively using magnetic signals. With super-resolution capabilities, MMG has the potential to improve spatial resolution and, in the meantime, address the limitations of EMG. This article discusses the challenges in developing magnetic sensors for MMG, including sensor design and technology advancements that allow for more specific recordings, targeting of individual motor units, and reduction of magnetic noise. In addition, we cover the motor unit behavior and activation pattern, an overview of magnetic sensing technologies, and evaluations of wearable, non-invasive magnetic sensors for MMG.

Physical activity in young children across developmental and health states: the ActiveCHILD study.

Background: Evidence about physical activity of young children across developmental and health states is very limited. Using data from an inclusive UK cohort, ActiveCHILD, we investigated relationships between objectively measured physical activity, child development, social context, and health-related quality of life (HRQoL). Methods: Children (12-36 months), purposively sampled across health pathways, developmental abilities, and sociodemographic factors, were recruited through thirteen National Health Service organisations in England. Data were collected from 07/2017 to 08/2019 on: weekly physical activity (3-7 days) using waist-worn accelerometer (ActiGraph 3GTX); sociodemographics, parent actions, child HRQoL, and child development using questionnaires; and child health conditions using clinical records. A data-driven, unsupervised method, called hidden semi-Markov model (HSMM) segmented the accelerometery data and provided estimates of the total time spent active (any intensity) and very active (greater intensity) for each child. Relationships with the explanatory factors were investigated using multiple linear regression. Findings: Physical activity data were obtained for 282 children (56% females, mean age 21 months, 37.5% with a health condition) covering all index of multiple deprivation deciles. The patterns of physical activity consisted of two daily peaks, children spending 6.44 (SD = 1.39) hours active (any intensity), of which 2.78 (SD = 1.38) hours very active, 91% meeting WHO guidelines. The model for total time active (any intensity) explained 24% of variance, with mobility capacity the strongest predictor (ß = 0.41). The model for time spent very active explained 59% of variance, with mobility capacity again the strongest predictor (ß = 0.76). There was no evidence of physical activity explaining HRQoL. Interpretation: The findings provide new evidence that young children across developmental states regularly achieve mainstream recommended physical activity levels and challenges the belief that children with development problems need lower expectations for daily physical activity compared to peers. Advancing the rights of all children to participate in physical activity requires inclusive, equally ambitious, expectations for all. Funding: Niina Kolehmainen, HEE/NIHR Integrated Clinical Academic Senior Clinical Lecturer, NIHR ICA-SCL-2015-01-00, was funded by the NIHR for this research project. Christopher Thornton, Olivia Craw, Laura Kudlek, and Laura Cutler were also funded from this award. Tim Rapley is a member of the NIHR Applied Research Collaboration North East and North Cumbria, with part of his time funded through the related award (NIHR200173). The views expressed in this publication are those of the author(s) and not necessarily those of the NIHR, NHS, or the UK Department of Health and Social Care. The work of Kianoush Nazarpour is supported by Engineering and Physical Sciences Research Council (EPSRC), under grant number EP/R004242/2.

Explainable and Robust Deep Forests for EMG-Force Modeling.

Machine and deep learning techniques have received increasing attentions in estimating finger forces from high-density surface electromyography (HDsEMG), especially for neural interfacing. However, most machine learning models are normally employed as block-box modules. Additionally, most previous models suffer from performance degradation when dealing with noisy signals. In this work, we propose to employ a forest ensemble model for HDsEMG-force modeling. Our model is explainable and robust against noise. Additionally, we explored the effect of increasing the depth of forest models in EMG-force modeling problems. We evaluated the performance of deep forests with a finger force estimation task. Training and testing data were acquired 3-25 days apart, approximating realistic scenarios. Results showed that deep forests significantly outperformed other models. With artificial signal distortion in 20% channels, deep forests also showed a higher robustness, with the error reduced from that of the baseline by 50% compared with all other models. We provided explanations for the proposed model using the mean decrease impurity (MDI) metric, revealing a strong correspondence between the model and physiology.

Using unsupervised machine learning to quantify physical activity from accelerometry in a diverse and rapidly changing population.

Accelerometers are widely used to measure physical activity behaviour, including in children. The traditional method for processing acceleration data uses cut points to define physical activity intensity, relying on calibration studies that relate the magnitude of acceleration to energy expenditure. However, these relationships do not generalise across diverse populations and hence they must be parametrised for each subpopulation (e.g., age groups) which is costly and makes studies across diverse populations and over time difficult. A data-driven approach that allows physical activity intensity states to emerge from the data, without relying on parameters derived from external populations, offers a new perspective on this problem and potentially improved results. We applied an unsupervised machine learning approach, namely a hidden semi-Markov model, to segment and cluster the raw accelerometer data recorded (using a waist-worn ActiGraph GT3X+) from 279 children (9-38 months old) with a diverse range of developmental abilities (measured using the Paediatric Evaluation of Disability Inventory-Computer Adaptive Testing measure). We benchmarked this analysis with the cut points approach, calculated using thresholds from the literature which had been validated using the same device and for a population which most closely matched ours. Time spent active as measured by this unsupervised approach correlated more strongly with PEDI-CAT measures of the child's mobility (R2: 0.51 vs 0.39), social-cognitive capacity (R2: 0.32 vs 0.20), responsibility (R2: 0.21 vs 0.13), daily activity (R2: 0.35 vs 0.24), and age (R2: 0.15 vs 0.1) than that measured using the cut points approach. Unsupervised machine learning offers the potential to provide a more sensitive, appropriate, and cost-effective approach to quantifying physical activity behaviour in diverse populations, compared to the current cut points approach. This, in turn, supports research that is more inclusive of diverse or rapidly changing populations.

Delaying feedback during pre-device training facilitates the retention of novel myoelectric skills: a laboratory and home-based study.

Objective.The objective of this study was to assess the impact of delayed feedback training on the retention of novel myoelectric skills, and to demonstrate the use of this training approach in the home environment.Approach.We trained limb-intact participants to use a motor learning-based upper-limb prosthesis control scheme called abstract decoding. A delayed feedback paradigm intended to prevent within-trial adaptation and to facilitate motor learning was used. We conducted two multi-day experiments. Experiment 1 was a laboratory-based study consisting of two groups trained over a 4 day period with concurrent or delayed feedback. An additional follow-up session took place after 18 days to assess the retention of motor skills. Experiment 2 was a home-based pilot study that took place over five consecutive days to investigate delayed feedback performance when using bespoke training structures.Main Results.Approximately 35 000 trials were collected across both experiments. Experiment 1 found that the retention of motor skills for the delayed feedback group was significantly better than that of their concurrent feedback counterparts. In addition, the delayed feedback group improved their retention of motor skills across days, whereas the concurrent feedback group did not. Experiment 2 demonstrated that by using a bespoke training protocol in an environment that is more conducive to learning, it is possible for participants to become highly accurate in the absence of feedback.Significance.These results show that with delayed feedback training, it is possible to retain novel myoelectric skills. Using abstract decoding participants can activate four distinct muscle patterns without using complex algorithms. The accuracy achieved in the pilot study supports the feasibility of motor learning-based upper-limb prosthesis control after home-based myoelectric training.

Wearable super-resolution muscle-machine interfacing.

Muscles are the actuators of all human actions, from daily work and life to communication and expression of emotions. Myography records the signals from muscle activities as an interface between machine hardware and human wetware, granting direct and natural control of our electronic peripherals. Regardless of the significant progression as of late, the conventional myographic sensors are still incapable of achieving the desired high-resolution and non-invasive recording. This paper presents a critical review of state-of-the-art wearable sensing technologies that measure deeper muscle activity with high spatial resolution, so-called super-resolution. This paper classifies these myographic sensors according to the different signal types (i.e., biomechanical, biochemical, and bioelectrical) they record during measuring muscle activity. By describing the characteristics and current developments with advantages and limitations of each myographic sensor, their capabilities are investigated as a super-resolution myography technique, including: (i) non-invasive and high-density designs of the sensing units and their vulnerability to interferences, (ii) limit-of-detection to register the activity of deep muscles. Finally, this paper concludes with new opportunities in this fast-growing super-resolution myography field and proposes promising future research directions. These advances will enable next-generation muscle-machine interfaces to meet the practical design needs in real-life for healthcare technologies, assistive/rehabilitation robotics, and human augmentation with extended reality.

Increasing Voluntary Myoelectric Training Time Through Game Design.

In virtual prosthetic training research, serious games have been investigated for over 30 years. However, few game design elements are used and assessed for their effect on the voluntary adherence and repetition of the performed task. We compared two game-based versions of an established myoelectric-controlled virtual prosthetic training task with an interface without game elements of the same task [for video, see (Garske, 2022)]. Twelve limb-intact participants were sorted into three groups of comparable ability and asked to perform the task as long as they were motivated. Following the task, they completed a questionnaire regarding their motivation and engagement in the task. The investigation established that participants in the game-based groups performed the task significantly longer when more game design elements were implemented in the task (medians of 6 vs. 9.5 vs. 14 blocks for groups with increasing number of different game design elements). The participants in the game-based versions were also more likely to end the task out of fatigue than for reasons of boredom or frustration, which was verified by a fatigue analysis of the myoelectric signal. We demonstrated that the utilization of game design methodically in virtual myoelectric training tasks can support adherence and duration of a virtual training, in the short-term. Whether such short-term enhanced engagement would lead to long-term adherence remains an open question.

Internet of Things for beyond-the-laboratory prosthetics research.

Research on upper-limb prostheses is typically laboratory-based. Evidence indicates that research has not yet led to prostheses that meet user needs. Inefficient communication loops between users, clinicians and manufacturers limit the amount of quantitative and qualitative data that researchers can use in refining their innovations. This paper offers a first demonstration of an alternative paradigm by which remote, beyond-the-laboratory prosthesis research according to user needs is feasible. Specifically, the proposed Internet of Things setting allows remote data collection, real-time visualization and prosthesis reprogramming through Wi-Fi and a commercial cloud portal. Via a dashboard, the user can adjust the configuration of the device and append contextual information to the prosthetic data. We evaluated this demonstrator in real-time experiments with three able-bodied participants. Results promise the potential of contextual data collection and system update through the internet, which may provide real-life data for algorithm training and reduce the complexity of send-home trials. This article is part of the theme issue 'Advanced neurotechnologies: translating innovation for health and well-being'.

Towards User-Centred Prosthetics Research Beyond the Laboratory.

The purpose of this study was to explore a range of perspectives on how academic research and clinical assessment of upper-limb prosthetics could happen in environments outside of laboratories and clinics, such as within peoples' homes. Two co-creation workshops were held, which included people who use upper limb prosthetic devices (hereafter called users), clinicians, academics, a policy stakeholder, and a representative from the upper-limb prosthetics industry (hereafter called professionals). The discussions during the workshops indicate that research and clinical assessment conducted remotely from a laboratory or clinic could inform future solutions that address user needs. Users were open to the idea of sharing sensor and contextual data from within their homes to external laboratories during research studies. However, this was dependent upon several considerations, such as choice and control over data collection. Regarding clinical assessment, users had reservations of how data may be used to inform future prosthetic prescriptions whilst, clinicians were concerned with resource implications and capacity to process user data. The paper presents findings of the discussions shared by participants during both workshops. The paper concludes with a conjecture that collecting sensor and contextual data from users within their home environment will contribute towards literature within the field, and potentially inform future care policies for upper limb prosthetics. The involvement of users during such studies will be critical and can be enabled via a co-creation approach. In the short term, this may be achieved through academic research studies, which may in the long term inform a framework for clinical in-home trials and clinical remote assessment.

How Fast Is Too Fast? Boundaries to the Perception of Electrical Stimulation of Peripheral Nerves.

Transcutaneous electrical stimulation is a promising technique for providing prosthetic hand users with information about sensory events. However, questions remain over how to design the stimulation paradigms to provide users the best opportunity to discriminate these events. Here, we investigate if the refractory period influences how the amplitude of the applied stimulus is perceived. Twenty participants completed a two-alternative forced choice experiment. We delivered two stimuli spaced between 250 ms to 450 ms apart (inter-stimulus-interval, isi). The participants reported which stimulus they perceived as strongest. Each stimulus consisted of either a single or paired pulse delivered transcutaneously. The inter-pulse interval (ipi) for the paired pulse stimuli varied between 6 and 10 ms. We found paired pulses with an ipi of 6 ms were perceived stronger than a single pulse less often than paired pulses with an ipi of 8 ms (p = 0.001) or 10 ms (p < 0.0001). Additionally, we found when the isi was 250 ms, participants were less likely to identify the paired pulse as strongest, than when the isi was 350 or 450 ms. This study emphasizes the importance of basing stimulation paradigms on the underlying neural physiology. The results indicate there is an upper limit to the commonly accepted notion that higher stimulation frequencies lead to stronger perception. If frequency is to be used to encode sensory events, then the results suggest stimulus paradigms should be designed using frequencies below 125 Hz.

Sensory Feedback for Upper-Limb Prostheses: Opportunities and Barriers.

The addition of sensory feedback to upper-limb prostheses has been shown to improve control, increase embodiment, and reduce phantom limb pain. However, most commercial prostheses do not incorporate sensory feedback due to several factors. This paper focuses on the major challenges of a lack of deep understanding of user needs, the unavailability of tailored, realistic outcome measures and the segregation between research on control and sensory feedback. The use of methods such as the Person-Based Approach and co-creation can improve the design and testing process. Stronger collaboration between researchers can integrate different prostheses research areas to accelerate the translation process.

Discrete Action Control for Prosthetic Digits.

We aim to develop a paradigm for simultaneous and independent control of multiple degrees of freedom (DOFs) for upper-limb prostheses. To that end, we introduce action control, a novel method to operate prosthetic digits with surface electromyography (EMG) based on multi-output, multi-class classification. At each time step, the decoder classifies movement intent for each controllable DOF into one of three categories: open, close, or stall (i.e., no movement). We implemented a real-time myoelectric control system using this method and evaluated it by running experiments with one unilateral and two bilateral amputees. Participants controlled a six-DOF bar interface on a computer display, with each DOF corresponding to a motor function available in multi-articulated prostheses. We show that action control can significantly and systematically outperform the state-of-the-art method of position control via multi-output regression in both task- and non-task-related measures. Using the action control paradigm, improvements in median task performance over regression-based control ranged from 20.14% to 62.32% for individual participants. Analysis of a post-experimental survey revealed that all participants rated action higher than position control in a series of qualitative questions and expressed an overall preference for the former. Action control has the potential to improve the dexterity of upper-limb prostheses. In comparison with regression-based systems, it only requires discrete instead of real-valued ground truth labels, typically collected with motion tracking systems. This feature makes the system both practical in a clinical setting and also suitable for bilateral amputation. This work is the first demonstration of myoelectric digit control in bilateral upper-limb amputees. Further investigation and pre-clinical evaluation are required to assess the translational potential of the method.

Recalibration of myoelectric control with active learning.

Introduction: Improving the robustness of myoelectric control to work over many months without the need for recalibration could reduce prosthesis abandonment. Current approaches rely on post-hoc error detection to verify the certainty of a decoder's prediction using predefined threshold value. Since the decoder is fixed, the performance decline over time is inevitable. Other approaches such as supervised recalibration and unsupervised self-recalibration entail limitations in scaling up and computational resources. The objective of this paper is to study active learning as a scalable, human-in-the-loop framework, to improve the robustness of myoelectric control. Method: Active learning and linear discriminate analysis methods were used to create an iterative learning process, to modify decision boundaries based on changes in the data. We simulated a real-time scenario. We exploited least confidence, smallest margin and entropy reduction sampling strategies in single and batch-mode sample selection. Optimal batch-mode sampling was considered using ranked batch-mode active learning. Results: With only 3.2 min of data carefully selected by the active learner, the decoder outperforms random sampling by 4-5 and ~2% for able-bodied and people with limb difference, respectively. We observed active learning strategies to systematically and significantly enhance the decoders adaptation while optimizing the amount of training data on a class-specific basis. Smallest margin and least confidence uncertainty were shown to be the most supreme. Discussion: We introduce for the first time active learning framework for long term adaptation in myoelectric control. This study simulates closed-loop environment in an offline manner and proposes a pipeline for future real-time deployment.

Combined Dynamic Time Warping and Spatiotemporal Attention for Myoelectric Control.

The success of pattern recognition based upper-limb prostheses control is linked to their ability to extract appropriate features from the electromyogram (EMG) signals. Traditional EMG feature extraction (FE) algorithms fail to extract spatial and inter-temporal information from the raw data, as they consider the EMG channels individually across a set of sliding windows with some degree of overlapping. To tackle these limitations, this paper presents a method that considers the spatial information of multi-channel EMG signals by utilising dynamic time warping (DTW). To satisfy temporal considerations, inspired by Long Short-Term Memory (LSTM) neural networks, our algorithm evolves the DTW feature representation across long and short-term components to capture the temporal dynamics of the EMG signal. As such the contribution of this paper is the development of a recursive spatio-temporal FE method, denoted as Recursive Temporal Warping (RTW). To investigate the performance of the proposed method, an offline EMG pattern recognition study with 53 movement classes performed by 10 subjects wearing 8 to 16 EMG channels was considered with the results compared against several conventional as well as deep learning-based models. We show that the use of the RTW can reduce classification errors significantly, paving the way for future real-time implementation.

Long-Term Myoelectric Training with Delayed Feedback in the Home Environment.

Myoelectric prosthesis users typically do not receive immediate feedback from their device. They must be able to consistently produce distinct muscle activations in the absence of augmented feedback. In previous experiments, abstract decoding has provided real-time visual feedback for closed loop control. It is unclear if the performance in those experiments was due to short-term adaptation or motor learning. To test if similar performance could be reached without short-term adaptation, we trained participants with a delayed feedback paradigm. Feedback was delayed until after the ~1.5 s trial was completed. Three participants trained for five days in their home environments, completing a cumulative total of 4920 trials. Participants became highly accurate while receiving no real-time feedback of their control input. They were also able to retain performance gains across days. This strongly suggests that abstract decoding with delayed feedback facilitates motor learning, enabling four class control without immediate feedback.

Flexible Piezoelectric Sensors for Miniaturized Sonomyography.

Sonomyography refers to the measurement of muscle activity with an ultrasonic transducer. It is a candidate modality for applications in diagnosis of muscle conditions, rehabilitation engineering and prosthesis control as an alternative to electromyography. We propose a mechanically-flexible piezoelectric sonomyography transducer. Simulating different components of the transducer, using COMSOL Multiphysics® software, we analyze various electromechanical parameters, such as von Mises stress and charge accumulation. Our findings on modelling of a single-element device, comprised of a PZT-5H layer of thickness 66µm, with a polymer substrate (E = 2.5 GPa), demonstrate optimal flexibility and charge accumulation for sonomyography. The addition of Polyimide and PMMA (Polymethyl methacrylate) as an acoustic matching layer and an acoustic lens, respectively, allowed for adequate energy transfer to the medium, whilst still maintaining good mechanical properties. In addition, preliminary ultrasound transmission simulations (200 kHz to 30 MHz) showed the importance of the aspect ratio of the device and how there is a need for further studies on it. The development of such a technology could be of great use within the healthcare sector, not only due to its ability to provide highly accurate and varied real-time muscle data, but also because of the range of applications that could benefit from its use.