Adaptive assistive robotics: a framework for triadic collaboration between humans and robots.

Robots and other assistive technologies have a huge potential to help society in domains ranging from factory work to healthcare. However, safe and effective control of robotic agents in these environments is complex, especially when it involves close interactions and multiple actors. We propose an effective framework for optimizing the behaviour of robots and complementary assistive technologies in systems comprising a mix of human and technological agents with numerous high-level goals. The framework uses a combination of detailed biomechanical modelling and weighted multi-objective optimization to allow for the fine tuning of robot behaviours depending on the specification of the task at hand. We illustrate our framework via two case studies across assisted living and rehabilitation scenarios, and conduct simulations and experiments of triadic collaboration in practice. Our results indicate a marked benefit to the triadic approach, showing the potential to improve outcome measures for human agents in robot-assisted tasks.

A Haptic Sleeve as a Method of Mechanotactile Feedback Restoration for Myoelectric Hand Prosthesis Users.

Current myoelectric upper limb prostheses do not restore sensory feedback, impairing fine motor control. Mechanotactile feedback restoration with a haptic sleeve may rectify this problem. This randomised crossover within-participant controlled study aimed to assess a prototype haptic sleeve's effect on routine grasping tasks performed by eight able-bodied participants. Each participant completed 15 repetitions of the three tasks: Task 1-normal grasp, Task 2-strong grasp and Task 3-weak grasp, using visual, haptic, or combined feedback All data were collected in April 2021 in the Scottish Microelectronics Centre, Edinburgh, UK. Combined feedback correlated with significantly higher grasp success rates compared to the vision alone in Task 1 (p < 0.0001), Task 2 (p = 0.0057), and Task 3 (p = 0.0170). Similarly, haptic feedback was associated with significantly higher grasp success rates compared to vision in Task 1 (p < 0.0001) and Task 2 (p = 0.0015). Combined feedback correlated with significantly lower energy expenditure compared to visual feedback in Task 1 (p < 0.0001) and Task 3 (p = 0.0003). Likewise, haptic feedback was associated with significantly lower energy expenditure compared to the visual feedback in Task 1 (p < 0.0001), Task 2 (p < 0.0001), and Task 3 (p < 0.0001). These results suggest that mechanotactile feedback provided by the haptic sleeve effectively augments grasping and reduces its energy expenditure.

HapFIC: An Adaptive Force/Position Controller for Safe Environment Interaction in Articulated Systems.

Haptic interaction is essential for the dynamic dexterity of animals, which seamlessly switch from an impedance to an admittance behaviour using the force feedback from their proprioception. However, this ability is extremely challenging to reproduce in robots, especially when dealing with complex interaction dynamics, distributed contacts, and contact switching. Current model-based controllers require accurate interaction modelling to account for contacts and stabilise the interaction. In this manuscript, we propose an adaptive force/position controller that exploits the fractal impedance controller's passivity and non-linearity to execute a finite search algorithm using the force feedback signal from the sensor at the end-effector. The method is computationally inexpensive, opening the possibility to deal with distributed contacts in the future. We evaluated the architecture in physics simulation and showed that the controller can robustly control the interaction with objects of different dynamics without violating the maximum allowable target forces or causing numerical instability even for very rigid objects. The proposed controller can also autonomously deal with contact switching and may find application in multiple fields such as legged locomotion, rehabilitation and assistive robotics.

Multi-Grip Classification-Based Prosthesis Control With Two EMG-IMU Sensors.

In the field of upper-limb myoelectric prosthesis control, the use of statistical and machine learning methods has been long proposed as a means of enabling intuitive grip selection and actuation. Recently, this paradigm has found its way toward commercial adoption. Machine learning-based prosthesis control typically relies on the use of a large number of electrodes. Here, we propose an end-to-end strategy for multi-grip, classification-based prosthesis control using only two sensors, comprising electromyography (EMG) electrodes and inertial measurement units (IMUs). We emphasize the importance of accurately estimating posterior class probabilities and rejecting predictions made with low confidence, so as to minimize the rate of unintended prosthesis activations. To that end, we propose a confidence-based error rejection strategy using grip-specific thresholds. We evaluate the efficacy of the proposed system with real-time pick and place experiments using a commercial multi-articulated prosthetic hand and involving 12 able-bodied and two transradial (i.e., below-elbow) amputee participants. Results promise the potential for deploying intuitive, classification-based multi-grip control in existing upper-limb prosthetic systems subject to small modifications.

Effect of User Practice on Prosthetic Finger Control With an Intuitive Myoelectric Decoder.

Machine learning-based myoelectric control is regarded as an intuitive paradigm, because of the mapping it creates between muscle co-activation patterns and prosthesis movements that aims to simulate the physiological pathways found in the human arm. Despite that, there has been evidence that closed-loop interaction with a classification-based interface results in user adaptation, which leads to performance improvement with experience. Recently, there has been a focus shift toward continuous prosthesis control, yet little is known about whether and how user adaptation affects myoelectric control performance in dexterous, intuitive tasks. We investigate the effect of short-term adaptation with independent finger position control by conducting real-time experiments with 10 able-bodied and two transradial amputee subjects. We demonstrate that despite using an intuitive decoder, experience leads to significant improvements in performance. We argue that this is due to the lack of an utterly natural control scheme, which is mainly caused by differences in the anatomy of human and artificial hands, movement intent decoding inaccuracies, and lack of proprioception. Finally, we extend previous work in classification-based and wrist continuous control by verifying that offline analyses cannot reliably predict real-time performance, thereby reiterating the importance of validating myoelectric control algorithms with real-time experiments.

Bio-inspired design and validation of the Efficient Lockable Spring Ankle (ELSA) prosthesis.

Over the last decade, active lower-limb prostheses demonstrated their ability to restore a physiological gait for transfemoral amputees by supplying the required positive energy balance during daily life locomotion activities. However, the added-value of such devices is significantly impacted by their limited energetic autonomy, excessive weight and cost, thus preventing their full appropriation by the users. There is thus a strong incentive to produce active yet affordable, lightweight and energy efficient devices. To address these issues, we developed the ELSA (Efficient Lockable Spring Ankle) prosthesis embedding both a lockable parallel spring and a series elastic actuator, tailored to the walking dynamics of a sound ankle. The first contribution of this paper concerns the developement of a bio-inspired, lightweight and stiffness-adjustable parallel spring, comprising an energy efficient ratchet and pawl mechanism with servo actuation. The second contribution is the addition of a complementary rope-driven series elastic actuator to generate the active push-off. The system produces a sound ankle torque pattern during flat ground walking. Up to 50% of the peak torque is generated passively at a negligible energetic cost (0.1 J/stride). By design, the total system is lightweight (1.2kg) and low cost.

Analytic Model for Quadruped Locomotion Task-Space Planning.

Despite the extensive presence of the legged locomotion in animals, it is extremely challenging to be reproduced with robots. Legged locomotion is an dynamic task which benefits from a planning that takes advantage of the gravitational pull on the system. However, the computational cost of such optimization rapidly increases with the complexity of kinematic structures, rendering impossible real-time deployment in unstructured environments. This paper proposes a simplified method that can generate desired centre of mass and feet trajectory for quadrupeds. The model describes a quadruped as two bipeds connected via their centres of mass, and it is based on the extension of an algebraic bipedal model that uses the topology of the gravitational attractor to describe bipedal locomotion strategies. The results show that the model generates trajectories that agrees with previous studies. The model will be deployed in the future as seed solution for whole-body trajectory optimization in the attempt to reduce the computational cost and obtain real-time planning of complex action in challenging environments.

Continuous Versus Discrete Simultaneous Control of Prosthetic Fingers.

Modern, commercially available hand prostheses offer the potential of individual digit control. However, this feature is often not utilized due to the lack of a robust scheme for finger motion estimation from surface electromyographic (EMG) measurements. Regression methods have been proposed to achieve closed-loop finger position, velocity, or force control. In this paper, we propose an alternative approach, based on open-loop action-based control, which could be achieved through simultaneous finger motion classification. We compare the efficacy of continuous closed-loop and discrete open-loop control on the task of controlling the five degrees of actuation (DOAs) of a dexterous robotic hand. Eight normally-limbed subjects were instructed to teleoperate the hand using a data glove and the two control schemes under investigation in order to match target postures presented to them on a screen as closely as possible. Results indicate that, firstly, the performance of the two control methods is comparable and, secondly, that experience can lead to significant performance improvement over time, regardless of the method used. These results suggest that prosthetic finger control in a continuous space can be potentially achieved by means of myoelectric classification and discrete, action-based control and hence encourage further research in this direction.

Causes of Performance Degradation in Non-invasive Electromyographic Pattern Recognition in Upper Limb Prostheses.

Surface Electromyography (EMG)-based pattern recognition methods have been investigated over the past years as a means of controlling upper limb prostheses. Despite the very good reported performance of myoelectric controlled prosthetic hands in lab conditions, real-time performance in everyday life conditions is not as robust and reliable, explaining the limited clinical use of pattern recognition control. The main reason behind the instability of myoelectric pattern recognition control is that EMG signals are non-stationary in real-life environments and present a lot of variability over time and across subjects, hence affecting the system's performance. This can be the result of one or many combined changes, such as muscle fatigue, electrode displacement, difference in arm posture, user adaptation on the device over time and inter-subject singularity. In this paper an extensive literature review is performed to present the causes of the drift of EMG signals, ways of detecting them and possible techniques to counteract for their effects in the application of upper limb prostheses. The suggested techniques are organized in a table that can be used to recognize possible problems in the clinical application of EMG-based pattern recognition methods for upper limb prosthesis applications and state-of-the-art methods to deal with such problems.

Effectively Quantifying the Performance of Lower-Limb Exoskeletons Over a Range of Walking Conditions.

Exoskeletons and other wearable robotic devices have a wide range of potential applications, including assisting patients with walking pathologies, acting as tools for rehabilitation, and enhancing the capabilities of healthy humans. However, applying these devices effectively in a real-world setting can be challenging, as the optimal design features and control commands for an exoskeleton are highly dependent on the current user, task and environment. Consequently, robust metrics and methods for quantifying exoskeleton performance are required. This work presents an analysis of walking data collected for healthy subjects walking with an active pelvis exoskeleton over three assistance scenarios and five walking contexts. Spatial and temporal, kinematic, kinetic and other novel dynamic gait metrics were compared to identify which metrics exhibit desirable invariance properties, and so are good candidates for use as a stability metric over varying walking conditions. Additionally, using a model-based approach, the average metabolic power consumption was calculated for a subset of muscles crossing the hip, knee and ankle joints, and used to analyse how the energy-reducing properties of an exoskeleton are affected by changes in walking context. The results demonstrated that medio-lateral centre of pressure displacement and medio-lateral margin of stability exhibit strong invariance to changes in walking conditions. This suggests that these dynamic gait metrics are optimised in human gait and are potentially suitable metrics for optimising in an exoskeleton control paradigm. The effectiveness of the exoskeleton at reducing human energy expenditure was observed to increase when walking on an incline, where muscles aiding in hip flexion were assisted, but decrease when walking at a slow speed. These results underline the need for adaptive control algorithms for exoskeletons if they are to be used in varied environments.

Improved prosthetic hand control with concurrent use of myoelectric and inertial measurements.

BACKGROUND: Myoelectric pattern recognition systems can decode movement intention to drive upper-limb prostheses. Despite recent advances in academic research, the commercial adoption of such systems remains low. This limitation is mainly due to the lack of classification robustness and a simultaneous requirement for a large number of electromyogram (EMG) electrodes. We propose to address these two issues by using a multi-modal approach which combines surface electromyography (sEMG) with inertial measurements (IMs) and an appropriate training data collection paradigm. We demonstrate that this can significantly improve classification performance as compared to conventional techniques exclusively based on sEMG signals. METHODS: We collected and analyzed a large dataset comprising recordings with 20 able-bodied and two amputee participants executing 40 movements. Additionally, we conducted a novel real-time prosthetic hand control experiment with 11 able-bodied subjects and an amputee by using a state-of-the-art commercial prosthetic hand. A systematic performance comparison was carried out to investigate the potential benefit of incorporating IMs in prosthetic hand control. RESULTS: The inclusion of IM data improved performance significantly, by increasing classification accuracy (CA) in the offline analysis and improving completion rates (CRs) in the real-time experiment. Our findings were consistent across able-bodied and amputee subjects. Integrating the sEMG electrodes and IM sensors within a single sensor package enabled us to achieve high-level performance by using on average 4-6 sensors. CONCLUSIONS: The results from our experiments suggest that IMs can form an excellent complimentary source signal for upper-limb myoelectric prostheses. We trust that multi-modal control solutions have the potential of improving the usability of upper-extremity prostheses in real-life applications.

Self-tuning bistable parametric feedback oscillator: Near-optimal amplitude maximization without model information.

Theory predicts that parametrically excited oscillators, tuned to operate under resonant condition, are capable of large-amplitude oscillation useful in diverse applications, such as signal amplification, communication, and analog computation. However, due to amplitude saturation caused by nonlinearity, lack of robustness to model uncertainty, and limited sensitivity to parameter modulation, these oscillators require fine-tuning and strong modulation to generate robust large-amplitude oscillation. Here we present a principle of self-tuning parametric feedback excitation that alleviates the above-mentioned limitations. This is achieved using a minimalistic control implementation that performs (i) self-tuning (slow parameter adaptation) and (ii) feedback pumping (fast parameter modulation), without sophisticated signal processing past observations. The proposed approach provides near-optimal amplitude maximization without requiring model-based control computation, previously perceived inevitable to implement optimal control principles in practical application. Experimental implementation of the theory shows that the oscillator self-tunes itself near to the onset of dynamic bifurcation to achieve extreme sensitivity to small resonant parametric perturbations. As a result, it achieves large-amplitude oscillations by capitalizing on the effect of nonlinearity, despite substantial model uncertainties and strong unforeseen external perturbations. We envision the present finding to provide an effective and robust approach to parametric excitation when it comes to real-world application.

Target Uncertainty Mediates Sensorimotor Error Correction.

Human movements are prone to errors that arise from inaccuracies in both our perceptual processing and execution of motor commands. We can reduce such errors by both improving our estimates of the state of the world and through online error correction of the ongoing action. Two prominent frameworks that explain how humans solve these problems are Bayesian estimation and stochastic optimal feedback control. Here we examine the interaction between estimation and control by asking if uncertainty in estimates affects how subjects correct for errors that may arise during the movement. Unbeknownst to participants, we randomly shifted the visual feedback of their finger position as they reached to indicate the center of mass of an object. Even though participants were given ample time to compensate for this perturbation, they only fully corrected for the induced error on trials with low uncertainty about center of mass, with correction only partial in trials involving more uncertainty. The analysis of subjects' scores revealed that participants corrected for errors just enough to avoid significant decrease in their overall scores, in agreement with the minimal intervention principle of optimal feedback control. We explain this behavior with a term in the loss function that accounts for the additional effort of adjusting one's response. By suggesting that subjects' decision uncertainty, as reflected in their posterior distribution, is a major factor in determining how their sensorimotor system responds to error, our findings support theoretical models in which the decision making and control processes are fully integrated.

Towards low-dimensionsal proportional myoelectric control.

One way of enhancing the dexterity of powered myoelectric prostheses is via proportional and simultaneous control of multiple degrees-of-freedom (DOFs). Recently, it has been demonstrated that the reconstruction of finger movement is feasible by using features of the surface electromyogram (sEMG) signal. In such paradigms, the number of predictors and target variables is usually large, and strong correlations are present in both the input and output domains. Synergistic patterns in the sEMG space have been previously exploited to facilitate kinematics decoding. In this work, we propose a framework for simultaneous input-output dimensionality reduction based on the generalized eigenvalue problem formulation of multiple linear regression (MLR). We demonstrate that the proposed methodology outperforms simultaneous input-output dimensionality reduction based on principal component analysis (PCA), while the prediction accuracy of the full rank regression (FRR) method can be achieved by using only a few relevant dimensions.

On the origins of suboptimality in human probabilistic inference.

Humans have been shown to combine noisy sensory information with previous experience (priors), in qualitative and sometimes quantitative agreement with the statistically-optimal predictions of Bayesian integration. However, when the prior distribution becomes more complex than a simple Gaussian, such as skewed or bimodal, training takes much longer and performance appears suboptimal. It is unclear whether such suboptimality arises from an imprecise internal representation of the complex prior, or from additional constraints in performing probabilistic computations on complex distributions, even when accurately represented. Here we probe the sources of suboptimality in probabilistic inference using a novel estimation task in which subjects are exposed to an explicitly provided distribution, thereby removing the need to remember the prior. Subjects had to estimate the location of a target given a noisy cue and a visual representation of the prior probability density over locations, which changed on each trial. Different classes of priors were examined (Gaussian, unimodal, bimodal). Subjects' performance was in qualitative agreement with the predictions of Bayesian Decision Theory although generally suboptimal. The degree of suboptimality was modulated by statistical features of the priors but was largely independent of the class of the prior and level of noise in the cue, suggesting that suboptimality in dealing with complex statistical features, such as bimodality, may be due to a problem of acquiring the priors rather than computing with them. We performed a factorial model comparison across a large set of Bayesian observer models to identify additional sources of noise and suboptimality. Our analysis rejects several models of stochastic behavior, including probability matching and sample-averaging strategies. Instead we show that subjects' response variability was mainly driven by a combination of a noisy estimation of the parameters of the priors, and by variability in the decision process, which we represent as a noisy or stochastic posterior.

Generalizability of EMG decoding using local field potentials.

Motor cortical local field potentials (LFPs) have been successfully used to decode both kinematics and kinetics of arm movement. For future clinically viable prostheses, however, brain activity decoders will have to generalize well under a wide spectrum of behavioral conditions. This property has not yet been demonstrated clearly. Here, we provide evidence for the first time, that an LFP-based electromyogram (EMG) decoder can generalize reasonably well across two different types of behavior. We implanted intracortical microelectrode arrays in the primary motor (M1) and ventral pre-motor (PMv) cortices of a rhesus macaque, and recorded LFP and EMG activity from arm and hand muscles of the contralateral forelimb during a two-dimensional (2-D) centre-out isometric wrist torque task (TT), and during free reach and grasp behavior (FB). Selected temporal and spectral features of the LFP signals were used to train EMG decoders using data from both types of behavior separately. We assessed the decoding performance for both within- and across-task cases. The average achieved generalization score was 65 ± 20%, while in many cases individual scores reached 100%.

Dynamic obstacle avoidance using Bayesian Occupancy Filter and approximate inference.

The goal of this paper is to solve the problem of dynamic obstacle avoidance for a mobile platform using the stochastic optimal control framework to compute paths that are optimal in terms of safety and energy efficiency under constraints. We propose a three-dimensional extension of the Bayesian Occupancy Filter (BOF) (Coué et al. Int. J. Rob. Res. 2006, 25, 19-30) to deal with the noise in the sensor data, improving the perception stage. We reduce the computational cost of the perception stage by estimating the velocity of each obstacle using optical flow tracking and blob filtering. While several obstacle avoidance systems have been presented in the literature addressing safety and optimality of the robot motion separately, we have applied the approximate inference framework to this problem to combine multiple goals, constraints and priors in a structured way. It is important to remark that the problem involves obstacles that can be moving, therefore classical techniques based on reactive control are not optimal from the point of view of energy consumption. Some experimental results, including comparisons against classical algorithms that highlight the advantages, are presented.

Internal representations of temporal statistics and feedback calibrate motor-sensory interval timing.

Humans have been shown to adapt to the temporal statistics of timing tasks so as to optimize the accuracy of their responses, in agreement with the predictions of Bayesian integration. This suggests that they build an internal representation of both the experimentally imposed distribution of time intervals (the prior) and of the error (the loss function). The responses of a Bayesian ideal observer depend crucially on these internal representations, which have only been previously studied for simple distributions. To study the nature of these representations we asked subjects to reproduce time intervals drawn from underlying temporal distributions of varying complexity, from uniform to highly skewed or bimodal while also varying the error mapping that determined the performance feedback. Interval reproduction times were affected by both the distribution and feedback, in good agreement with a performance-optimizing Bayesian observer and actor model. Bayesian model comparison highlighted that subjects were integrating the provided feedback and represented the experimental distribution with a smoothed approximation. A nonparametric reconstruction of the subjective priors from the data shows that they are generally in agreement with the true distributions up to third-order moments, but with systematically heavier tails. In particular, higher-order statistical features (kurtosis, multimodality) seem much harder to acquire. Our findings suggest that humans have only minor constraints on learning lower-order statistical properties of unimodal (including peaked and skewed) distributions of time intervals under the guidance of corrective feedback, and that their behavior is well explained by Bayesian decision theory.

Continuous evolution of statistical estimators for optimal decision-making.

In many everyday situations, humans must make precise decisions in the presence of uncertain sensory information. For example, when asked to combine information from multiple sources we often assign greater weight to the more reliable information. It has been proposed that statistical-optimality often observed in human perception and decision-making requires that humans have access to the uncertainty of both their senses and their decisions. However, the mechanisms underlying the processes of uncertainty estimation remain largely unexplored. In this paper we introduce a novel visual tracking experiment that requires subjects to continuously report their evolving perception of the mean and uncertainty of noisy visual cues over time. We show that subjects accumulate sensory information over the course of a trial to form a continuous estimate of the mean, hindered only by natural kinematic constraints (sensorimotor latency etc.). Furthermore, subjects have access to a measure of their continuous objective uncertainty, rapidly acquired from sensory information available within a trial, but limited by natural kinematic constraints and a conservative margin for error. Our results provide the first direct evidence of the continuous mean and uncertainty estimation mechanisms in humans that may underlie optimal decision making.

The role of feed-forward and feedback processes for closed-loop prosthesis control.

BACKGROUND: It is widely believed that both feed-forward and feed-back mechanisms are required for successful object manipulation. Open-loop upper-limb prosthesis wearers receive no tactile feedback, which may be the cause of their limited dexterity and compromised grip force control. In this paper we ask whether observed prosthesis control impairments are due to lack of feedback or due to inadequate feed-forward control. METHODS: Healthy subjects were fitted with a closed-loop robotic hand and instructed to grasp and lift objects of different weights as we recorded trajectories and force profiles. We conducted three experiments under different feed-forward and feed-back configurations to elucidate the role of tactile feedback (i) in ideal conditions, (ii) under sensory deprivation, and (iii) under feed-forward uncertainty. RESULTS: (i) We found that subjects formed economical grasps in ideal conditions. (ii) To our surprise, this ability was preserved even when visual and tactile feedback were removed. (iii) When we introduced uncertainty into the hand controller performance degraded significantly in the absence of either visual or tactile feedback. Greatest performance was achieved when both sources of feedback were present. CONCLUSIONS: We have introduced a novel method to understand the cognitive processes underlying grasping and lifting. We have shown quantitatively that tactile feedback can significantly improve performance in the presence of feed-forward uncertainty. However, our results indicate that feed-forward and feed-back mechanisms serve complementary roles, suggesting that to improve on the state-of-the-art in prosthetic hands we must develop prostheses that empower users to correct for the inevitable uncertainty in their feed-forward control.