Simultaneous and Proportional Control of Wrist and Hand Movements Based on a Neural-Driven Musculoskeletal Model.

Human-machine interfaces (HMIs) based on electromyography (EMG) signals have been developed for simultaneous and proportional control (SPC) of multiple degrees of freedom (DoFs). The EMG-driven musculoskeletal model (MM) has been used in HMIs to predict human movements in prosthetic and robotic control. However, the neural information extracted from surface EMG signals may be distorted due to their limitations. With the development of high density (HD) EMG decomposition, accurate neural drive signals can be extracted from surface EMG signals. In this study, a neural-driven MM was proposed to predict metacarpophalangeal (MCP) joint flexion/extension and wrist joint flexion/extension. Ten non-disabled subjects (male) were recruited and tested. Four 64-channel electrode grids were attached to four forearm muscles of each subject to record the HD EMG signals. The joint angles were recorded synchronously. The acquired HD EMG signals were decomposed to extract the motor unit (MU) discharge for estimating the neural drive, which was then used as the input to the MM to calculate the muscle activation and predict the joint movements. The Pearson's correlation coefficient (r) and the normalized root mean square error (NRMSE) between the predicted joint angles and the measured joint angles were calculated to quantify the estimation performance. Compared to the EMG-driven MM, the neural-driven MM attained higher r values and lower NRMSE values. Although the results were limited to an offline application and to a limited number of DoFs, they indicated that the neural-driven MM outperforms the EMG-driven MM in prediction accuracy and robustness. The proposed neural-driven MM for HMI can obtain more accurate neural commands and may have great potential for medical rehabilitation and robot control.

Clean visual field reconstruction in robot-assisted laparoscopic surgery based on dynamic prediction.

Robot-assisted minimally invasive surgery has been broadly employed in complicated operations. However, the multiple surgical instruments may occupy a large amount of visual space in complex operations performed in narrow spaces, which affects the surgeon's judgment on the shape and position of the lesion as well as the course of its adjacent vessels/lacunae. In this paper, a surgical scene reconstruction method is proposed, which involves the tracking and removal of surgical instruments and the dynamic prediction of the obscured region. For tracking and segmentation of instruments, the image sequences are preprocessed by a modified U-Net architecture composed of a pre-trained ResNet101 encoder and a redesigned decoder. Also, the segmentation boundaries of the instrument shafts are extended using image filtering and a real-time index mask algorithm to achieve precise localization of the obscured elements. For predicting the deformation of soft tissues, a soft tissue deformation prediction algorithm is proposed based on dense optical flow gravitational field and entropy increase, which can achieve local dynamic visualization of the surgical scene by integrating image morphological operations. Finally, the preliminary experiments and the pre-clinical evaluation were presented to demonstrate the performance of the proposed method. The results show that the proposed method can provide the surgeon with a clean and comprehensive surgical scene, reconstruct the course of important vessels/lacunae, and avoid inadvertent injuries.

Eliciting tactile sensations in the hand through non-invasive proximal nerve stimulation: a feasibility study.

Recently, non-invasive proximal nerve stimulation has been widely investigated to restore tactile sensations. It has been demonstrated that tactile sensations in the hand could be elicited by nerve stimulation on the upper arm. However, it is still unknown whether tactile sensations could be elicited by stimulation at a proximal location close to the neck. In this study, non-invasive proximal nerve stimulation tests were performed to elicit tactile sensations in the hand of subjects. Six Ag/AgCl gel electrodes (2 × 3) were placed on the supraclavicular fossa where the proximal parts of the brachial plexus nerves were located. Then, fifteen potential electrode pairs were tested to explore whether tactile sensations could be elicited by non-invasive proximal nerve stimulation. Eight able-bodied subjects (male) were recruited to participate in the test. The stimulated sensation regions in the hand and the sensory intensity were reported and recorded during the experiment. The results demonstrated that the tactile sensations in various regions in the hand could be elicited through non-invasive nerve stimulation at the proximal location close to the neck.

Bio-robotics research for non-invasive myoelectric neural interfaces for upper-limb prosthetic control: a 10-year perspective review.

A decade ago, a group of researchers from academia and industry identified a dichotomy between the industrial and academic state-of-the-art in upper-limb prosthesis control, a widely used bio-robotics application. They proposed that four key technical challenges, if addressed, could bridge this gap and translate academic research into clinically and commercially viable products. These challenges are unintuitive control schemes, lack of sensory feedback, poor robustness and single sensor modality. Here, we provide a perspective review on the research effort that occurred in the last decade, aiming at addressing these challenges. In addition, we discuss three research areas essential to the recent development in upper-limb prosthetic control research but were not envisioned in the review 10 years ago: deep learning methods, surface electromyogram decomposition and open-source databases. To conclude the review, we provide an outlook into the near future of the research and development in upper-limb prosthetic control and beyond.

Effect of Subcutaneous Muscle Displacement of Flexor Carpi Radialis on Surface Electromyography.

Changes in joint angle can change the position and orientation of muscle fibers relative to the surface EMG electrode. Our previous study has shown that EMG patterns can identify hand/wrist movements with a greater degree of classification accuracy (CA) when muscle contractions involve a change in the joint angle. The results of this study suggest that changes in the position of the muscle relative to the recording electrode can influence the properties of the recorded EMG signals, however, this was not directly quantified. The present study aims to further investigate the effect of subcutaneous muscle displacement caused by the changes in joint angle on surface EMG signals. Nine able-bodied subjects were tested. The subjects were instructed to perform wrist flexion at five different joint angles (0, 20, 40, 60, and 80) with the same level of muscle contraction. EMG signals and ultrasound images were acquired from the flexor carpi radialis (FCR) simultaneously. Time and frequency domain analysis was adopted to extract features from the EMG signals. The subcutaneous muscle displacement of the FCR relative to the skin surface was measured from the ultrasound images. Spearmans rank correlation coefficient was employed to analyze the correlation between the subcutaneous muscle displacement and the EMG signals. The results showed the subcutaneous muscle displacement of the FCR measured by the ultrasound images was 1 cm when the wrist joint angle changed from 0 to 80. There was a positive relationship between the subcutaneous muscle displacement and the mean absolute value (MAV) ( rs = 0.896 ) and median frequency (MF) ( rs = 0.849 ) extracted from the EMG signals. The results demonstrated that subcutaneous muscle displacement associated with wrist angle change had a significant effect on FCR EMG signals. This property might have a positive effect on the CA of dynamic tasks.

A High-Performance Transmitarray Antenna with Thin Metasurface for 5G Communication Based on PSO (Particle Swarm Optimization).

A 5G metasurface (MS) transmitarray (TA) feed by compact-antenna array with the performance of high gain and side-lobe level (SLL) reduction is presented. The proposed MS has two identical metallic layers etched on both sides of the dielectric substrate and four fixed vias connecting two metallic layers that works at 28 GHz to increase the transmission phase shift range. The proposed planar TA consisting of unit cells with different dimensional information can simulate the function as an optical lens according to the Fermat's principle, so the quasi-spherical wave emitted by the compact Potter horn antenna at the virtual focal point will transform to the quasi-plane wave by the phase-adjustments. Then, the particle swarm optimization (PSO) is introduced to optimize the phase distribution on the TA to decrease the SLL further. It is found that the optimized TA could achieve 27 dB gain at 28 GHz, 11.8% 3 dB gain bandwidth, -30 dB SLL, and aperture efficiency of 23% at the operating bandwidth of 27.5-29.5 GHz, which performs better than the nonoptimized one. The advanced particularities of this optimized TA including low cost, low profile, and easy to configure make it great potential in paving the way to 5G communication and radar system.

Evoking haptic sensations in the foot through high-density transcutaneous electrical nerve stimulations.

OBJECTIVE: Evoking haptic sensation on upper limb amputees via peripheral nerve stimulation has been investigated intensively in the past decade, but related studies involving lower limb amputees are limited. This study aimed to evaluate the feasibility of using non-invasive transcutaneous electrical nerve stimulation to evoke haptic sensation along the phantom limb of the amputated foot of transtibial amputees. APPROACH: A high-density electrode grid (4 × 4) was placed over the skin surface above the distal branching of the sciatic, tibial, and common peroneal nerves. We hypothesized that electrical stimulation delivered to distinct electrode pairs created unique electric fields, which can activate selective sets of sensory axons innervating different skin regions of the foot. Five transtibial amputee subjects (three unilateral and two bilateral) and one able-bodied subject were tested by scanning all possible electrode pair combinations. MAIN RESULTS: All subjects reported various haptic percepts at distinct regions along the foot with each corresponding to specific electrode pairs. These results demonstrated the capability of our non-invasive nerve stimulation method to evoke haptic sensations in the foot of transtibial amputees and the able-bodied subject. SIGNIFICANCE: The outcomes contribute important knowledge and evidence regarding missing tactile sensation in the foot of lower limb amputees and might also facilitate future development of strategies to manage phantom pain and enhance embodiment of prosthetic legs in the future.

Comparing EMG-Based Human-Machine Interfaces for Estimating Continuous, Coordinated Movements.

Electromyography (EMG)-based interfaces are trending toward continuous, simultaneous control with multiple degrees of freedom. Emerging methods range from data-driven approaches to biomechanical model-based methods. However, there has been no direct comparison between these two types of continuous EMG-based interfaces. The aim of this study was to compare a musculoskeletal model (MM) with two data-driven approaches, linear regression (LR) and artificial neural network (ANN), for predicting continuous wrist and hand motions for EMG-based interfaces. Six able-bodied subjects and one transradial amputee subject performed (missing) metacarpophalangeal (MCP) and wrist flexion/extension, simultaneously or independently, while four EMG signals were recorded from forearm muscles. To add variation to the EMG signals, the subjects repeated the MCP and wrist motions at various upper extremity postures. For each subject, the EMG signals collected from the neutral posture were used to build the EMG interfaces; the EMG signals collected from all postures were used to evaluate the interfaces. The performance of the interface was quantified by Pearson's correlation coefficient (r) and the normalized root mean square error (NRMSE) between measured and estimated joint angles. The results demonstrated that the MM predicted movements more accurately, with higher r values and lower NRMSE, than either LR or ANN. Similar results were observed in the transradial amputee. Additionally, the variation in r across postures, an indicator of reliability against posture changes, was significantly lower (better) for the MM than for either LR or ANN. Our findings suggest that incorporating musculoskeletal knowledge into EMG-based human-machine interfaces could improve the estimation of continuous, coordinated motion.

A Reliable Multi-User EMG Interface Based on A Generic-Musculoskeletal Model against Loading Weight Changes.

The reliability of myoelectric control is important to ensure the performance of prostheses during daily use. Recently, we proposed a multi-user neural-machine interface based on a generic musculoskeletal model to simultaneously and continuously estimate flexion/extension movements at the metacarpophalangeal (MCP) and wrist joints from surface electromyography (EMG) signals. Our previous results demonstrated that the multi-user EMG interface was reliable against upper limb posture changes. However, the reliability of the interface against different loading weights, which is an important factor that would decrease the performance of myoelectric control and be tested during the occupational therapy for myoelectric prosthesis users, is still unclear. In this study, we aimed to evaluate the reliability of the generic model over different loading weights. Four able-bodied subjects were tested in this study. Subjects performed a virtual hand/wrist posture matching task with three different loading weights (no weight, 1.25 Lbs, and 2.5 Lbs). All subjects accomplished all the assigned virtual tasks. The on-line experimental results showed that performance with different loading weights was very close. The results demonstrated that the multi-user EMG interface was reliable against the different loading weights, indicating it has potential to promote the myoelectric control into clinical applications.

Myoelectric Control Based on a Generic Musculoskeletal Model: Toward a Multi-User Neural-Machine Interface.

This paper aimed to develop a novel electromyography (EMG)-based neural-machine interface (NMI) that is user-generic for continuously predicting coordinated motion betweenmuscle contractionmetacarpophalangeal (MCP) and wrist flexion/extension. The NMI requires a minimum calibration procedure that only involves capturing maximal voluntary muscle contraction for themonitoredmuscles for individual users. At the center of the NMI is a user-generic musculoskeletal model based on the experimental data collected from six able-bodied (AB) subjects and nine different upper limb postures. The generic model was evaluated on-line on both AB subjects and a transradial amputee. The subjectswere instructed to performa virtual hand/wrist posture matching task with different upper limb postures. The on-line performanceof the genericmodelwas also compared with that of the musculoskeletal model customized to each individual user (called "specific model"). All subjects accomplished the assigned virtual tasks while using the user-generic NMI, although the AB subjects produced better performance than the amputee subject. Interestingly, compared with the specific model, the generic model produced comparable completion time, a reduced number of overshoots, and improved path efficiency in the virtual hand/wrist posture matching task. The results suggested that it is possible to design an EMG-driven NMI based on a musculoskeletalmodelthat could fit multiple users, including upper limb amputees, for predicting coordinated MCP and wrist motion. The present new method might address the challenges of existing advanced EMG-based NMI that require frequent and lengthy customization and calibration. Our future research will focus on evaluating the developed NMI for powered prosthetic arms.

Comparing Surface and Intramuscular Electromyography for Simultaneous and Proportional Control Based on a Musculoskeletal Model: A Pilot Study.

Simultaneous and proportional control (SPC) of neural-machine interfaces uses magnitudes of smoothed electromyograms (EMG) as control inputs. Though surface EMG (sEMG) electrodes are common for clinical neural-machine interfaces, intramuscular EMG (iEMG) electrodes may be indicated in some circumstances (e.g., for controlling many degrees of freedom). However, differences in signal characteristics between sEMG and iEMG may influence SPC performance. We conducted a pilot study to determine the effect of electrode type (sEMG and iEMG) on real-time task performance with SPC based on a novel 2-degree-of-freedom EMG-driven musculoskeletal model of the wrist and hand. Four able-bodied subjects and one transradial amputee performed a virtual posture matching task with either sEMG or iEMG. There was a trend of better task performance with sEMG than iEMG for both able-bodied and amputee subjects, though the difference was not statistically significant. Thus, while iEMG may permit targeted recording of EMG, its signal characteristics may not be as ideal for SPC as those of sEMG. The tradeoff between recording specificity and signal characteristics is an important consideration for development and clinical implementation of SPC for neural-machine interfaces.

Myoelectric Control Based on A Generic Musculoskeletal Model: Towards A Multi-User Neural-Machine Interface.

This study aimed to develop a novel electromyography (EMG)-based neural-machine interface (NMI) that is user-generic for continuously predicting coordinated motion between metacarpophalangeal (MCP) and wrist flexion/extension. The NMI requires a minimum calibration procedure that only involves capturing maximal voluntary muscle contraction for the monitored muscles for individual users. At the center of the NMI is a user-generic musculoskeletal model based on the experimental data collected from 6 able-bodied (AB) subjects and 9 different upper limb postures. The generic model was evaluated on-line on both AB subjects and a transradial amputee. The subjects were instructed to perform a virtual hand/wrist posture matching task with different upper limb postures. The on-line performance of the generic model was also compared with that of the musculoskeletal model customized to each individual user (called "specific model"). All subjects accomplished the assigned virtual tasks while using the user-generic NMI, although the AB subjects produced better performance than the amputee subject. Interestingly, compared to the specific model, the generic model produced comparable completion time, a reduced number of overshoots, and improved path efficiency in the virtual hand/wrist posture matching task. The results suggested that it is possible to design an EMG-driven NMI based on a musculoskeletal model that could fit multiple users, including upper limb amputees, for predicting coordinated MCP and wrist motion. The present new method might address the challenges of existing advanced EMG-based NMI that require frequent and lengthy customization and calibration. Our future research will focus on evaluating the developed NMI for powered prosthetic arms.

Transcranial direct current stimulation versus user training on improving online myoelectric control for amputees.

OBJECTIVE: Transcranial direct current stimulation (tDCS) and user training (UT) are two types of methods to improve myoelectric control performance for amputees. In this study, we compared the independent effect between tDCS and UT, and investigated the combined effect of tDCS and UT. APPROACH: An online paradigm of simultaneous and proportional control (SPC) based on electromyography (EMG) was adopted. The proposed experiments were conducted on six naïve unilateral trans-radial amputees. The subjects each received three types of 20 min interventions: active tDCS with motor training (tDCS + UT), active tDCS with quiet sitting (tDCS), and sham tDCS with motor training (UT). The interventions were applied at one week intervals in a randomized order. The subjects performed online control of a feedback arrow with two degrees of freedom (DoFs) to accomplish target reaching motor tasks in pre-sessions and post-sessions. We compared the performance, measured by completion rate, completion time, and efficiency coefficient, between pre-sessions and post-sessions. MAIN RESULTS: The results showed that the intervention tDCS + UT and tDCS significantly improved the online SPC performance (i.e. improved the completion rate; reduced the completion time; and improved the efficiency coefficient), while intervention UT did not significantly change the performance. The results also showed that the online SPC performance after intervention tDCS + UT and tDCS was not significantly different, but both were significantly better than that after intervention UT. SIGNIFICANCE: tDCS could be an effective intervention to improve the online SPC performance in a short time.

Improving robustness against electrode shift of high density EMG for myoelectric control through common spatial patterns.

BACKGROUND: Most prosthetic myoelectric control studies have concentrated on low density (less than 16 electrodes, LD) electromyography (EMG) signals, due to its better clinical applicability and low computation complexity compared with high density (more than 16 electrodes, HD) EMG signals. Since HD EMG electrodes have been developed more conveniently to wear with respect to the previous versions recently, HD EMG signals become an alternative for myoelectric prostheses. The electrode shift, which may occur during repositioning or donning/doffing of the prosthetic socket, is one of the main reasons for degradation in classification accuracy (CA). METHODS: HD EMG signals acquired from the forearm of the subjects were used for pattern recognition-based myoelectric control in this study. Multiclass common spatial patterns (CSP) with two types of schemes, namely one versus one (CSP-OvO) and one versus rest (CSP-OvR), were used for feature extraction to improve the robustness against electrode shift for myoelectric control. Shift transversal (ST1 and ST2) and longitudinal (SL1 and SL2) to the direction of the muscle fibers were taken into consideration. We tested nine intact-limb subjects for eleven hand and wrist motions. The CSP features (CSP-OvO and CSP-OvR) were compared with three commonly used features, namely time-domain (TD) features, time-domain autoregressive (TDAR) features and variogram (Variog) features. RESULTS: Compared with the TD features, the CSP features significantly improved the CA over 10 % in all shift configurations (ST1, ST2, SL1 and SL2). Compared with the TDAR features, a. the CSP-OvO feature significantly improved the average CA over 5 % in all shift configurations; b. the CSP-OvR feature significantly improved the average CA in shift configurations ST1, SL1 and SL2. Compared with the Variog features, the CSP features significantly improved the average CA in longitudinal shift configurations (SL1 and SL2). CONCLUSION: The results demonstrated that the CSP features significantly improved the robustness against electrode shift for myoelectric control with respect to the commonly used features.

A structurally decoupled mechanism for measuring wrist torque in three degrees of freedom.

The wrist joint is a critical part of the human body for movement. Measuring the torque of the wrist with three degrees of freedom (DOFs) is important in some fields, including rehabilitation, biomechanics, ergonomics, and human-machine interfacing. However, the particular structure of the wrist joint makes it difficult to measure the torque in all three directions simultaneously. This work develops a structurally decoupled instrument for measuring and improving the measurement accuracy of 3-DOF wrist torque during isometric contraction. Three single-axis torque sensors were embedded in a customized mechanical structure. The dimensions and components of the instrument were designed based on requirement of manufacturability. A prototype of the instrument was machined, assembled, integrated, and tested. The results show that the structurally decoupled mechanism is feasible for acquiring wrist torque data in three directions either independently or simultaneously. As a case study, we use the device to measure wrist torques concurrently with electromyography signal acquisition in preparation for simultaneous and proportional myoelectric control of prostheses.

Improving Myoelectric Control for Amputees through Transcranial Direct Current Stimulation.

Most prosthetic myoelectric control studies have shown good performance for unimpaired subjects. However, performance is generally unacceptable for amputees. The primary problem is the poor quality of electromyography (EMG) signals of amputees compared with healthy individuals. To improve clinical performance of myoelectric control, this study explored transcranial direct current stimulation (tDCS) to modulate brain activity and enhance EMG quality. We tested six unilateral transradial amputees by applying active and sham anodal tDCS separately on two different days. Surface EMG signals were acquired from the affected and intact sides for 11 hand and wrist motions in the pre-tDCS and post-tDCS sessions. Autoregression coefficients and linear discriminant analysis classifiers were used to process the EMG data for pattern recognition of the 11 motions. For the affected side, active anodal tDCS significantly reduced the average classification error rate (CER) by 10.1%, while sham tDCS had no such effect. For the intact side, the average CER did not change on the day of sham tDCS but increased on the day of active tDCS. These results demonstrated that tDCS could modulate brain function and improve EMG-based classification performance for amputees. It has great potential in dramatically reducing the length of learning process of amputees for effectively using myoelectrically controlled multifunctional prostheses.

Residuals of autoregressive model providing additional information for feature extraction of pattern recognition-based myoelectric control.

Myoelectric control based on pattern recognition has been studied for several decades. Autoregressive (AR) features are one of the mostly used feature extraction methods among myoelectric control studies. Almost all previous studies only used the AR coefficients without the residuals of AR model for classification. However, the residuals of AR model contain important amplitude information of the electromyography (EMG) signals. In this study, we added the residuals to the AR features (AR+re) and compared its performance with the classical sixth-order AR coefficients. We tested six unilateral transradial amputees and eight able-bodied subjects for eleven hand and wrist motions. The classification accuracy (CA) of the intact side for amputee subjects and the right hand for able-bodied subjects showed that the CA of AR+re features was slightly but significantly higher than that of classical AR features (p = 0.009), which meant that residuals could provide additional information to classical AR features for classification. Interestingly, the CA of the affected side for amputee subjects showed that there was no significant difference between the CA of AR+re features and classical AR features (p > 0.05). We attributed this to the fact that the amputee subjects could not use their affected side to produce consistent EMG patterns as their intact side or the dominant hand of the able-bodied subjects. Since the residuals were already available when the AR coefficients were computed, the results of this study suggested adding the residuals to classical AR features to potentially improve the performance of pattern recognition-based myoelectric control.

Can transcranial direct current stimulation enhance performance of myoelectric control for multifunctional prosthesis?

Pattern recognition based myoelectric control has been studied by many researchers. However, the classification accuracy was pretty low for amputees towards multifunctional prosthesis control in practice. In this work, a novel method of transcranial direct current stimulation (tDCS) which can modulate brain activity was used to enhance performance for myoelectric prosthesis control. The pilot study was conducted on three able-bodied subjects and one transradial amputee. Surface electromyography (EMG) signals were acquired from both arms when performing eleven hand and wrist motions in pre-tDCS and post-tDCS sessions. Time domain (TD) features and linear discriminant analysis (LDA) classifier were adopted to process EMG. For the non-dominant hand of the healthy subjects, active anodal tDCS of the contralateral primary motor cortex was able to significantly improve average classification accuracy by 3.82% (p <; 0.05), while sham tDCS could not have such effect (p > 0.05). For amputated (phantom) hand of the amputee, active anodal tDCS was able to significantly improve average classification accuracy by 12.56%, while sham tDCS could not have such effect. For the dominant hand and intact hand, the average classification accuracies were stable and not significantly improved using either active tDCS or sham tDCS. The results show that tDCS is a powerful noninvasive method to modulate brain function and enhance EMG classification performance especially for the amputated hand towards multifunctional prosthesis control. The method proposed has a huge potential to promote EMG pattern recognition based control scheme to clinical application.

A linear model for simultaneously and proportionally estimating wrist kinematics from emg during mirrored bilateral movements.

This paper presents a linear model for simultaneous and proportional estimation of the two degree-of-freedoms (DOFs) wrist angle positions with surface electromyography (EMG). A 5th order state-space model was used to estimate wrist kinematics from 4-channel surface EMG signals of the contralateral forearm during mirrored bilateral movements without motion constraints. The EMG signal from each of the three limbed normal subjects was collected along with each angle position in two DOFs from both of the arms, with motion parameters tested including the radial/ulnar deviation and flexion/extension of the wrist. The estimation performance was in the range 0.787-0.885 (R(2) index) for the two DOFs in three limbed normal subjects. The results show that wrist kinematics can be estimated in 2 DOFs by state-space models with relative high accuracy compared with the results reported previously. The method proposed, as requiring only kinematics measured from the contralateral wrist, is potentially available for a unilateral amputee in simultaneous and proportional control of DOFs in powered upper limb prostheses.