Measuring and monitoring skill learning in closed-loop myoelectric hand prostheses using speed-accuracy tradeoffs.

Objective.Closed-loop myoelectric prostheses, which combine supplementary sensory feedback and electromyography (EMG) based control, hold the potential to narrow the divide between natural and bionic hands. The use of these devices, however, requires dedicated training. Therefore, it is crucial to develop methods that quantify how users acquire skilled control over their prostheses to effectively monitor skill progression and inform the development of interfaces that optimize this process.Approach.Building on theories of skill learning in human motor control, we measured speed-accuracy tradeoff functions (SAFs) to comprehensively characterize learning-induced changes in skill-as opposed to merely tracking changes in task success across training-facilitated by a closed-loop interface that combined proportional control and EMG feedback. Sixteen healthy participants and one individual with a transradial limb loss participated in a three-day experiment where they were instructed to perform the box-and-blocks task using a timed force-matching paradigm at four specified speeds to reach two target force levels, such that the SAF could be determined.Main results.We found that the participants' accuracy increased in a similar way across all speeds we tested. Consequently, the shape of the SAF remained similar across days, at both force levels. Further, we observed that EMG feedback enabled participants to improve their motor execution in terms of reduced trial-by-trial variability, a hallmark of skilled behavior. We then fit a power law model of the SAF, and demonstrated how the model parameters could be used to identify and monitor changes in skill.Significance.We comprehensively characterized how an EMG feedback interface enabled skill acquisition, both at the level of task performance and movement execution. More generally, we believe that the proposed methods are effective for measuring and monitoring user skill progression in closed-loop prosthesis control.

Contrasting action and posture coding with hierarchical deep neural network models of proprioception.

Biological motor control is versatile, efficient, and depends on proprioceptive feedback. Muscles are flexible and undergo continuous changes, requiring distributed adaptive control mechanisms that continuously account for the body's state. The canonical role of proprioception is representing the body state. We hypothesize that the proprioceptive system could also be critical for high-level tasks such as action recognition. To test this theory, we pursued a task-driven modeling approach, which allowed us to isolate the study of proprioception. We generated a large synthetic dataset of human arm trajectories tracing characters of the Latin alphabet in 3D space, together with muscle activities obtained from a musculoskeletal model and model-based muscle spindle activity. Next, we compared two classes of tasks: trajectory decoding and action recognition, which allowed us to train hierarchical models to decode either the position and velocity of the end-effector of one's posture or the character (action) identity from the spindle firing patterns. We found that artificial neural networks could robustly solve both tasks, and the networks' units show tuning properties similar to neurons in the primate somatosensory cortex and the brainstem. Remarkably, we found uniformly distributed directional selective units only with the action-recognition-trained models and not the trajectory-decoding-trained models. This suggests that proprioceptive encoding is additionally associated with higher-level functions such as action recognition and therefore provides new, experimentally testable hypotheses of how proprioception aids in adaptive motor control.

Estimating speed-accuracy trade-offs to evaluate and understand closed-loop prosthesis interfaces.

Objective. Closed-loop prosthesis interfaces, which combine electromyography (EMG)-based control with supplementary feedback, represent a promising direction for developing the next generation of bionic limbs. However, we still lack an understanding of how users utilize these interfaces and how to evaluate competing solutions. In this study, we used the framework of speed-accuracy trade-off functions (SAF) to understand, evaluate, and compare the performance of two closed-loop user-prosthesis interfaces.Approach. Ten able-bodied participants and an amputee performed a force-matching task in a functional box-and-block setup at three different speeds. All participants were subjected to both interfaces in a crossover study design with a 1 week washout period. Importantly, both interfaces used direct proportional control but differed in the feedback provided to the participant (EMG feedback vs. Force feedback). We estimated the SAFs afforded by the two interfaces and sought to understand how the participants planned and executed the task under the various conditions.Main results. We found that execution speed significantly influenced performance, and that EMG feedback afforded better overall performance, especially at medium speeds. Notably, we found that there was a difference in the SAF between the two interfaces, with EMG feedback enabling participants to attain higher accuracies faster than Force feedback. Furthermore, both interfaces enabled participants to develop flexible control policies, while EMG feedback also afforded participants the ability to generate smoother, more repeatable EMG commands.Significance. Overall, the results indicate that the performance of closed-loop prosthesis interfaces depends critically on the feedback approach and execution speed. This study showed that the SAF framework could be used to reveal the differences between feedback approaches, which might not have been detected if the assessment was performed at a single speed. Therefore, we argue that it is important to consider the speed-accuracy trade-offs to rigorously evaluate and compare user-prosthesis interfaces.

The impact of objective functions on control policies in closed-loop control of grasping force with a myoelectric prosthesis.

Objective.Supplemental sensory feedback for myoelectric prostheses can provide both psychosocial and functional benefits during prosthesis control. However, the impact of feedback depends on multiple factors and there is insufficient understanding about the fundamental role of such feedback in prosthesis use. The framework of human motor control enables us to systematically investigate the user-prosthesis control loop. In this study, we explore how different task objectives such as speed and accuracy shape the control policy developed by participants in a prosthesis force-matching task.Approach.Participants were randomly assigned to two groups that both used identical electromyography control interface and prosthesis force feedback, through vibrotactile stimulation, to perform a prosthesis force-matching task. However, the groups received different task objectives specifying speed and accuracy demands. We then investigated the control policies developed by the participants. To this end, we not only evaluated how successful or fast participants were but also analyzed the behavioral strategies adopted by the participants to obtain such performance gains.Main results.First, we observed that participants successfully integrated supplemental prosthesis force feedback to develop both feedforward and feedback control policies, as demanded by the task objectives. We then observed that participants who first developed a (slow) feedback policy were quickly able to adapt their policy to more stringent speed demands, by switching to a combined feedforward-feedback control strategy. However, the participants who first developed a (fast) feedforward policy were not able to change their control policy and adjust to greater accuracy demands.Significance.Overall, the results signify how the framework of human motor control can be applied to study the role of feedback in user-prosthesis interaction. The results also reveal the utility of training prosthesis users to integrate supplemental feedback into their state estimation by designing training protocols that encourage the development of combined feedforward and feedback policy.

Action representation in the mouse parieto-frontal network.

The posterior parietal cortex (PPC) and frontal motor areas comprise a cortical network supporting goal-directed behaviour, with functions including sensorimotor transformations and decision making. In primates, this network links performed and observed actions via mirror neurons, which fire both when individuals perform an action and when they observe the same action performed by a conspecific. Mirror neurons are believed to be important for social learning, but it is not known whether mirror-like neurons occur in similar networks in other social species, such as rodents, or if they can be measured in such models using paradigms where observers passively view a demonstrator. Therefore, we imaged Ca2+ responses in PPC and secondary motor cortex (M2) while mice performed and observed pellet-reaching and wheel-running tasks, and found that cell populations in both areas robustly encoded several naturalistic behaviours. However, neural responses to the same set of observed actions were absent, although we verified that observer mice were attentive to performers and that PPC neurons responded reliably to visual cues. Statistical modelling also indicated that executed actions outperformed observed actions in predicting neural responses. These results raise the possibility that sensorimotor action recognition in rodents could take place outside of the parieto-frontal circuit, and underscore that detecting socially-driven neural coding depends critically on the species and behavioural paradigm used.

DeepLabCut: markerless pose estimation of user-defined body parts with deep learning.

Quantifying behavior is crucial for many applications in neuroscience. Videography provides easy methods for the observation and recording of animal behavior in diverse settings, yet extracting particular aspects of a behavior for further analysis can be highly time consuming. In motor control studies, humans or other animals are often marked with reflective markers to assist with computer-based tracking, but markers are intrusive, and the number and location of the markers must be determined a priori. Here we present an efficient method for markerless pose estimation based on transfer learning with deep neural networks that achieves excellent results with minimal training data. We demonstrate the versatility of this framework by tracking various body parts in multiple species across a broad collection of behaviors. Remarkably, even when only a small number of frames are labeled (~200), the algorithm achieves excellent tracking performance on test frames that is comparable to human accuracy.