Functionally Adaptive Myosite Selection Using High-Density sEMG for Upper Limb Myoelectric Prostheses.

OBJECTIVE: Our study defines a novel electrode placement method called Functionally Adaptive Myosite Selection (FAMS), as a tool for rapid and effective electrode placement during prosthesis fitting. We demonstrate a method for determining electrode placement that is adaptable towards individual patient anatomy and desired functional outcomes, agnostic to the type of classification model used, and provides insight into expected classifier performance without training multiple models. METHODS: FAMS relies on a separability metric to rapidly predict classifier performance during prosthesis fitting. RESULTS: The results show a predictable relationship between the FAMS metric and classifier accuracy (3.45%SE), allowing estimation of control performance with any given set of electrodes. Electrode configurations selected using the FAMS metric show improved control performance ( ) for target electrode counts compared to established methods when using an ANN classifier, and equivalent performance ( R2 ≥ .96) to previous top-performing methods on an LDA classifier, with faster convergence ( ). We used the FAMS method to determine electrode placement for two amputee subjects by using the heuristic to search through possible sets, and checking for saturation in performance vs electrode count. The resulting configurations that averaged 95.8% of the highest possible classification performance using a mean 25 number of electrodes (19.5% of the available sites). SIGNIFICANCE: FAMS can be used to rapidly approximate the tradeoffs between increased electrode count and classifier performance, a useful tool during prosthesis fitting.

Limb loading enhances skill transfer between augmented and physical reality tasks during limb loss rehabilitation.

BACKGROUND: Virtual and augmented reality (AR) have become popular modalities for training myoelectric prosthesis control with upper-limb amputees. While some systems have shown moderate success, it is unclear how well the complex motor skills learned in an AR simulation transfer to completing the same tasks in physical reality. Limb loading is a possible dimension of motor skill execution that is absent in current AR solutions that may help to increase skill transfer between the virtual and physical domains. METHODS: We implemented an immersive AR environment where individuals could operate a myoelectric virtual prosthesis to accomplish a variety of object relocation manipulations. Intact limb participants were separated into three groups, the load control (CGLD; [Formula: see text]), the AR control (CGAR; [Formula: see text]), and the experimental group (EG; [Formula: see text]). Both the CGAR and EG completed a 5-session prosthesis training protocol in AR while the CGLD performed simple muscle training. The EG attempted manipulations in AR while undergoing limb loading. The CGAR attempted the same manipulations without loading. All participants performed the same manipulations in physical reality while operating a real prosthesis pre- and post-training. The main outcome measure was the change in the number of manipulations completed during the physical reality assessments (i.e. completion rate). Secondary outcomes included movement kinematics and visuomotor behavior. RESULTS: The EG experienced a greater increase in completion rate post-training than both the CGAR and CGLD. This performance increase was accompanied by a shorter motor learning phase, the EG's performance saturating in less sessions of AR training than the CGAR. CONCLUSION: The results demonstrated that limb loading plays an important role in transferring complex motor skills learned in virtual spaces to their physical reality analogs. While participants who did not receive limb loading were able to receive some functional benefit from AR training, participants who received the loading experienced a greater positive change in motor performance with their performance saturating in fewer training sessions.

A Myoelectric Postural Control Algorithm for Persons With Transradial Amputations: A Consideration of Clinical Readiness.

BACKGROUND: The bottleneck in upper limb prosthetic design is the myoelectric control algorithm. Here we studied the clinical readiness of the myoelectric postural control algorithm in a laboratory setting with two trans-radial amputees using a commercially available prosthetic limb system. TECHNIQUE: The postural control algorithm was integrated into prosthetic limb systems using standard of care components. A comparison between a commercial state of the art system (the i-limb revolution state-based myoelectric controller) and the postural controller was performed with two people with trans-radial amputation using a self-contained prosthesis system. DISCUSSION: The performance using the i-limb revolution state-based controller versus the postural controller was mixed based on the Southampton Hand Assessment Procedure. The SHAP scores indicate that the postural controller with i-limb revolution provided an average of 66% of hand function compared to an intact limb. Future work will study the advantages of the postural control algorithm in everyday use.

Stable Responsive EMG Sequence Prediction and Adaptive Reinforcement With Temporal Convolutional Networks.

Prediction of movement intentions from electromyographic (EMG) signals is typically performed with a pattern recognition approach, wherein a short dataframe of raw EMG is compressed into an instantaneous feature-encoding that is meaningful for classification. However, EMG signals are time-varying, implying that a frame-wise approach may not sufficiently incorporate temporal context into predictions, leading to erratic and unstable prediction behavior. OBJECTIVE: We demonstrate that sequential prediction models and, specifically, temporal convolutional networks are able to leverage useful temporal information from EMG to achieve superior predictive performance. METHODS: We compare this approach to other sequential and frame-wise models predicting 3 simultaneous hand and wrist degrees-of-freedom from 2 amputee and 13 non-amputee human subjects in a minimally constrained experiment. We also compare these models on the publicly available Ninapro and CapgMyo amputee and non-amputee datasets. RESULTS: Temporal convolutional networks yield predictions that are more accurate and stable than frame-wise models, especially during inter-class transitions, with an average response delay of 4.6 ms and simpler feature-encoding. Their performance can be further improved with adaptive reinforcement training. SIGNIFICANCE: Sequential models that incorporate temporal information from EMG achieve superior movement prediction performance and these models allow for novel types of interactive training. CONCLUSIONS: Addressing EMG decoding as a sequential modeling problem will lead to enhancements in the reliability, responsiveness, and movement complexity available from prosthesis control systems.

Prosthesis with neuromorphic multilayered e-dermis perceives touch and pain.

The human body is a template for many state-of-the-art prosthetic devices and sensors. Perceptions of touch and pain are fundamental components of our daily lives that convey valuable information about our environment while also providing an element of protection from damage to our bodies. Advances in prosthesis designs and control mechanisms can aid an amputee's ability to regain lost function but often lack meaningful tactile feedback or perception. Through transcutaneous electrical nerve stimulation (TENS) with an amputee, we discovered and quantified stimulation parameters to elicit innocuous (non-painful) and noxious (painful) tactile perceptions in the phantom hand. Electroencephalography (EEG) activity in somatosensory regions confirms phantom hand activation during stimulation. We invented a multilayered electronic dermis (e-dermis) with properties based on the behavior of mechanoreceptors and nociceptors to provide neuromorphic tactile information to an amputee. Our biologically inspired e-dermis enables a prosthesis and its user to perceive a continuous spectrum from innocuous to noxious touch through a neuromorphic interface that produces receptor-like spiking neural activity. In a Pain Detection Task (PDT), we show the ability of the prosthesis and amputee to differentiate non-painful or painful tactile stimuli using sensory feedback and a pain reflex feedback control system. In this work, an amputee can use perceptions of touch and pain to discriminate object curvature, including sharpness. This work demonstrates possibilities for creating a more natural sensation spanning a range of tactile stimuli for prosthetic hands.

Limb Position Tolerant Pattern Recognition for Myoelectric Prosthesis Control with Adaptive Sparse Representations From Extreme Learning.

Myoelectric signals can be used to predict the intended movements of an amputee for prosthesis control. However, untrained effects like limb position changes influence myoelectric signal characteristics, hindering the ability of pattern recognition algorithms to discriminate among motion classes. Despite frequent and long training sessions, these deleterious conditional influences may result in poor performance and device abandonment. GOAL: We present a robust sparsity-based adaptive classification method that is significantly less sensitive to signal deviations resulting from untrained conditions. METHODS: We compare this approach in the offline and online contexts of untrained upper-limb positions for amputee and able-bodied subjects to demonstrate its robustness compared against other myoelectric classification methods. RESULTS: We report significant performance improvements () in untrained limb positions across all subject groups. SIGNIFICANCE: The robustness of our suggested approach helps to ensure better untrained condition performance from fewer training conditions. CONCLUSIONS: This method of prosthesis control has the potential to deliver real-world clinical benefits to amputees: better condition-tolerant performance, reduced training burden in terms of frequency and duration, and increased adoption of myoelectric prostheses.

Predictive trajectory estimation during rehabilitative tasks in augmented reality using inertial sensors.

This paper presents a wireless kinematic tracking framework used for biomechanical analysis during rehabilitative tasks in augmented and virtual reality. The framework uses low-cost inertial measurement units and exploits the rigid connections of the human skeletal system to provide egocentric position estimates of joints to centimeter accuracy. On-board sensor fusion combines information from three-axis accelerometers, gyroscopes, and magnetometers to provide robust estimates in real-time. Sensor precision and accuracy were validated using the root mean square error of estimated joint angles against ground truth goniometer measurements. The sensor network produced a mean estimate accuracy of 2.81° with 1.06° precision, resulting in a maximum hand tracking error of 7.06 cm. As an application, the network is used to collect kinematic information from an unconstrained object manipulation task in augmented reality, from which dynamic movement primitives are extracted to characterize natural task completion in N = 3 able-bodied human subjects. These primitives are then leveraged for trajectory estimation in both a generalized and a subject-specific scheme resulting in 0.187 cm and 0.161 cm regression accuracy, respectively. Our proposed kinematic tracking network is wireless, accurate, and especially useful for predicting voluntary actuation in virtual and augmented reality applications.

Targeted Transcutaneous Electrical Nerve Stimulation for Phantom Limb Sensory Feedback.

In this work, we investigated the use of noninvasive, targeted transcutaneous electrical nerve stimulation (TENS) of peripheral nerves to provide sensory feedback to two amputees, one with targeted sensory reinnervation (TSR) and one without TSR. A major step in developing a closed-loop prosthesis is providing the sense of touch back to the amputee user. We investigated the effect of targeted nerve stimulation amplitude, pulse width, and frequency on stimulation perception. We discovered that both subjects were able to reliably detect stimulation patterns with pulses less than 1 ms. We utilized the psychophysical results to produce a subject specific stimulation pattern using a leaky integrate and fire (LIF) neuron model from force sensors on a prosthetic hand during a grasping task. For the first time, we show that TENS is able to provide graded sensory feedback at multiple sites in both TSR and non-TSR amputees while using behavioral results to tune a neuromorphic stimulation pattern driven by a force sensor output from a prosthetic hand.

Electrode-shift Tolerant Myoelectric Movement-pattern Classification using Extreme Learning for Adaptive Sparse Representations.

Myoelectric signal patterns can be used to predict the intended movements of amputees for prosthesis activation. Real-world prosthesis use introduces a variety of unpredictable conditional influences on these patterns, hindering the performance of classification algorithms and potentially leading to device abandonment. We have discovered a state-of-the-art classification method which is significantly more tolerant to these conditional influences. In our prior work, we presented a robust sparsity-based adaptive classification method that is tolerant to pattern deviations resulting from untrained limb positions and the prosthesis load. Herein, we demonstrate that this method is tolerant to the shifting or misalignment of the contact-electrode array which occurs during prosthesis use. We demonstrate the robustness of this approach in untrained electrode-site locations for amputee and able-bodied subjects, and report significant performance improvements over conventional myoelectric pattern recognition approaches. By showing that a single, unified method is robust across a variety of real-world condition spaces, clinicians are more likely to incorporate this method into myoelectric prosthesis controllers, resulting in improved utility and increased adoption among amputee users.

Neuromimetic Event-Based Detection for Closed-Loop Tactile Feedback Control of Upper Limb Prostheses.

Upper limb amputees lack the valuable tactile sensing that helps provide context about the surrounding environment. Here we utilize tactile information to provide active touch feedback to a prosthetic hand. First, we developed fingertip tactile sensors for producing biomimetic spiking responses for monitoring contact, release, and slip of an object grasped by a prosthetic hand. We convert the sensor output into pulses, mimicking the rapid and slowly adapting spiking responses of receptor afferents found in the human body. Second, we designed and implemented two neuromimetic event-based algorithms, Compliant Grasping and Slip Prevention, on a prosthesis to create a local closed-loop tactile feedback control system (i.e. tactile information is sent to the prosthesis). Grasping experiments were designed to assess the benefit of this biologically inspired neuromimetic tactile feedback to a prosthesis. Results from able-bodied and amputee subjects show the average number of objects that broke or slipped during grasping decreased by over 50% and the average time to complete a grasping task decreased by at least 10% for most trials when comparing neuromimetic tactile feedback with no feedback on a prosthesis. Our neuromimetic method of closed-loop tactile sensing is a novel approach to improving the function of upper limb prostheses.

Biologically inspired multi-layered synthetic skin for tactile feedback in prosthetic limbs.

The human body offers a template for many state-of-the-art prosthetic devices and sensors. In this work, we present a novel, sensorized synthetic skin that mimics the natural multi-layered nature of mechanoreceptors found in healthy glabrous skin to provide tactile information. The multi-layered sensor is made up of flexible piezoresistive textiles that act as force sensitive resistors (FSRs) to convey tactile information, which are embedded within a silicone rubber to resemble the compliant nature of human skin. The top layer of the synthetic skin is capable of detecting small loads less than 5 N whereas the bottom sensing layer responds reliably to loads over 7 N. Finite element analysis (FEA) of a simplified human fingertip and the synthetic skin was performed. Results suggest similarities in behavior during loading. A natural tactile event is simulated by loading the synthetic skin on a prosthetic limb. Results show the sensors' ability to detect applied loads as well as the ability to simulate neural spiking activity based on the derivative and temporal differences of the sensor response. During the tactile loading, the top sensing layer responded 0.24 s faster than the bottom sensing layer. A synthetic biologically-inspired skin such as this will be useful for enhancing the functionality of prosthetic limbs through tactile feedback.

Limb-position robust classification of myoelectric signals for prosthesis control using sparse representations.

The fundamental objective in non-invasive myoelectric prosthesis control is to determine the user's intended movements from corresponding skin-surface recorded electromyographic (sEMG) activation signals as quickly and accurately as possible. Linear Discriminant Analysis (LDA) has emerged as the de facto standard for real-time movement classification due to its ease of use, calculation speed, and remarkable classification accuracy under controlled training conditions. However, performance of cluster-based methods like LDA for sEMG pattern recognition degrades significantly when real-world testing conditions do not resemble the trained conditions, limiting the utility of myoelectrically controlled prosthesis devices. We propose an enhanced classification method that is more robust to generic deviations from training conditions by constructing sparse representations of the input data dictionary comprised of sEMG time-frequency features. We apply our method in the context of upper-limb position changes to demonstrate pattern recognition robustness and improvement over LDA across discrete positions not explicitly trained. For single position training we report an accuracy improvement in untrained positions of 7.95%, p ≪ .001, in addition to significant accuracy improvements across all multiposition training conditions, p <; .001.

Outcomes and Perception of a Conventional and Alternative Myoelectric Control Strategy: A Study of Experienced and New Multiarticulating Hand Users.

Introduction: The development of multiarticulating hands holds the potential to restore lost function for upper-limb amputees. However, access to the full potential of commercialized devices is limited due to conventional control strategies for switching prosthesis modes, such as hand grips. For example, to switch grips in one conventional strategy, the prosthesis user must generate electromyogram (EMG) triggers (such as a cocontraction), which are cumbersome and nonintuitive. For this reason, alternative control strategies have emerged, which seek to facilitate grip switching. One specific application uses radio frequency identification (RFID) tags programmed with grip information. These tags can be placed on objects in the environment or carried on person. Upon approaching an RFID tag, the user's prosthesis reads the grip programmed on the tag and commands the hand into that grip. The purpose of this study was to compare the conventional strategy (using EMG triggers) with the alternative strategy (using RFID tags). Methods: The study evaluated three subjects: two users who actively use multiarticulating hands ("experienced" users) and one user who had never worn a multiarticulating hand ("new" user). Subjects were evaluated on two performance metrics: trigger completion time and the percentage of triggers that were successful on first attempt (first attempt success rate). Subjects also rated the difficulty, effort, and frustration with each strategy. Results: Results suggested faster trigger completion times with the EMG strategy for the experienced users and mixed results for the new user. Overall, the three subjects rated the RFID strategy as less difficult, tiring, and frustrating than the EMG strategy. Discussion and Conclusions: Continued studies with a larger subject pool are necessary to determine factors influencing performance and patient preference. This would allow identification of best strategies to access the full potential of new commercial devices. Still, the authors suggest that the synergistic use of both strategies can yield great benefits for both experienced and new multiarticulating hand users.

User training for pattern recognition-based myoelectric prostheses: improving phantom limb movement consistency and distinguishability.

We assessed the ability of four transradial amputees to control a virtual prosthesis capable of nine classes of movement both before and after a two-week training period. Subjects attended eight one-on-one training sessions that focused on improving the consistency and distinguishability of their hand and wrist movements using visual biofeedback from a virtual prosthesis. The virtual environment facilitated the precise quantification of three prosthesis control measures. During a final evaluation, the subject population saw an average increase in movement completion percentage from 70.8% to 99.0%, an average improvement in normalized movement completion time from 1.47 to 1.13, and an average increase in movement classifier accuracy from 77.5% to 94.4% (p<0.001). Additionally, all four subjects were reevaluated after eight elapsed hours without retraining the classifier, and all subjects demonstrated minimal decreases in performance. Our analysis of the underlying sources of improvement for each subject examined the sizes and separation of high-dimensional data clusters and revealed that each subject formed a unique and effective strategy for improving the consistency and/or distinguishability of his or her phantom limb movements. This is the first longitudinal study designed to examine the effects of user training in the implementation of pattern recognition-based myoelectric prostheses.

Tactile Feedback in Upper Limb Prosthetic Devices Using Flexible Textile Force Sensors.

Many upper limb amputees are faced with the difficult challenge of using a prosthesis that lacks tactile sensing. State of the art research caliber prosthetic hands are often equipped with sophisticated sensors that provide valuable information regarding the prosthesis and its surrounding environment. Unfortunately, most commercial prosthetic hands do not contain any tactile sensing capabilities. In this paper, a textile based tactile sensor system was designed, built, and evaluated for use with upper limb prosthetic devices. Despite its simplicity, we demonstrate the ability of the sensors to determine object contact and perturbations due to slip during a grasping task with a prosthetic hand. This suggests the use of low-cost, customizable, textile sensors as part of a closed-loop tactile feedback system for monitoring grasping forces specifically in an upper limb prosthetic device.

Evaluation of a noninvasive command scheme for upper-limb prostheses in a virtual reality reach and grasp task.

C5/C6 tetraplegic patients and transhumeral amputees may be able to use voluntary shoulder motion as command signals for a functional electrical stimulation system or transhumeral prosthesis. Stereotyped relationships, termed "postural synergies," among the shoulder, forearm, and wrist joints emerge during goal-oriented reaching and transport movements as performed by able-bodied subjects. Thus, the posture of the shoulder can potentially be used to infer the desired posture of the elbow and forearm joints during reaching and transporting movements. We investigated how well able-bodied subjects could learn to use a noninvasive command scheme based on inferences from these postural synergies to control a simulated transhumeral prosthesis in a virtual reality task. We compared the performance of subjects using the inferential command scheme (ICS) with subjects operating the simulated prosthesis in virtual reality according to complete motion tracking of their actual arm and hand movements. Initially, subjects performed poorly with the ICS but improved rapidly with modest amounts of practice, eventually achieving performance only slightly less than subjects using complete motion tracking. Thus, inferring the desired movement of distal joints from voluntary shoulder movements appears to be an intuitive and noninvasive approach for obtaining command signals for prostheses to restore reaching and grasping functions.

Utilizing Tactile Feedback for Biomimetic Grasping Control in Upper Limb Prostheses.

A biomimetic system for enhancing the control and reliability of grasping with prosthetic hands was designed and experimentally evaluated. Barometric pressure sensors as well as a force-sensitive resistor (FSR) were placed on a prosthetic hand to provide valuable tactile feedback. Contact and slip detection grip control algorithms were developed to interpret force signals for enhancing stable grasping. Recent advances in radio-frequency identification (RFID) technology enable the amputee to select between grip control strategies based on the desired object to be grasped. Experimental results indicate that the control algorithms are capable of utilizing real-time force responses to detect object contact as well as slip. By allowing the user to act as a high-level controller with RFID technology, a multi-faceted low-level controller that responds to tactile feedback can be developed for enhancing grasping functionality in prosthetic hands.

Improving Long Term Myoelectric Decoding, Using an Adaptive Classifier with Label Correction.

This study presents a novel adaptive myoelectric decoding algorithm for control of upper limb prosthesis. Myoelectric decoding algorithms are inherently subject to decay in decoding accuracy over time, which is caused by the changes occurring in the muscle signals. The proposed algorithm relies on an unsupervised and on demand update of the training set, and has been designed to adapt to both the slow and fast changes that occur in myoelectric signals. An update in the training data is used to counter the slow changes, whereas an update with label correction addresses the fast changes in the signals. We collected myoelectric data from an able bodied user for over four and a half hours, while the user performed repetitions of eight wrist movements. The major benefit of the proposed algorithm is the lower rate of decay in accuracy; it has a decay rate of 0.2 per hour as opposed to 3.3 for the non adaptive classifier. The results show that, long term decoding accuracy in EMG signals can be maintained over time, improving the performance and reliability of myoelectric prosthesis.