Experimental evaluation of the impact of sEMG interfaces in enhancing embodiment of virtual myoelectric prostheses.

INTRODUCTION: Despite recent technological advances that have led to sophisticated bionic prostheses, attaining embodied solutions still remains a challenge. Recently, the investigation of prosthetic embodiment has become a topic of interest in the research community, which deals with enhancing the perception of artificial limbs as part of users' own body. Surface electromyography (sEMG) interfaces have emerged as a promising technology for enhancing upper-limb prosthetic control. However, little is known about the impact of these sEMG interfaces on users' experience regarding embodiment and their interaction with different functional levels. METHODS: To investigate this aspect, a comparison is conducted among sEMG configurations with different number of sensors (4 and 16 channels) and different time delay. We used a regression algorithm to simultaneously control hand closing/opening and forearm pronation/supination in an immersive virtual reality environment. The experimental evaluation includes 24 able-bodied subjects and one prosthesis user. We assess functionality with the Target Achievement Control test, and the sense of embodiment with a metric for the users perception of self-location, together with a standard survey. RESULTS: Among the four tested conditions, results proved a higher subjective embodiment when participants used sEMG interfaces employing an increased number of sensors. Regarding functionality, significant improvement over time is observed in the same conditions, independently of the time delay implemented. CONCLUSIONS: Our work indicates that a sufficient number of sEMG sensors improves both, functional and subjective embodiment outcomes. This prompts discussion regarding the potential relationship between these two aspects present in bionic integration. Similar embodiment outcomes are observed in the prosthesis user, showing also differences due to the time delay, and demonstrating the influence of sEMG interfaces on the sense of agency.

Spatial and Temporal Analysis of Normal and Shear Forces During Grasping, Manipulation and Social Activities.

Extensive research has established and widely acknowledged the important contribution of human hand sensory receptors in providing tactile feedback. The absence of these receptors results in a poor perception of the environment, impairing our efficient manipulation skills. Although the literature emphasizes the significance of normal forces in human grasping, further investigations should point toward the role of shear forces in this process. This paper presents an analysis of human everyday grasping activities through the use of 20 three-axis magnetic soft skin force sensors, in the form of rings and bands, that measure both normal and shear forces. Our study includes twelve tasks that cover various grasping requirements. Results highlight the importance of spatial information and the usefulness of shear forces in the prediction of unexpected changes that can not be always observed in normal forces. Tactile sensing can ultimately be integrated into prosthetic and rehabilitation devices for improved control and potentially provide sensory feedback to the user.

6IMPOSE: bridging the reality gap in 6D pose estimation for robotic grasping.

6D pose recognition has been a crucial factor in the success of robotic grasping, and recent deep learning based approaches have achieved remarkable results on benchmarks. However, their generalization capabilities in real-world applications remain unclear. To overcome this gap, we introduce 6IMPOSE, a novel framework for sim-to-real data generation and 6D pose estimation. 6IMPOSE consists of four modules: First, a data generation pipeline that employs the 3D software suite Blender to create synthetic RGBD image datasets with 6D pose annotations. Second, an annotated RGBD dataset of five household objects was generated using the proposed pipeline. Third, a real-time two-stage 6D pose estimation approach that integrates the object detector YOLO-V4 and a streamlined, real-time version of the 6D pose estimation algorithm PVN3D optimized for time-sensitive robotics applications. Fourth, a codebase designed to facilitate the integration of the vision system into a robotic grasping experiment. Our approach demonstrates the efficient generation of large amounts of photo-realistic RGBD images and the successful transfer of the trained inference model to robotic grasping experiments, achieving an overall success rate of 87% in grasping five different household objects from cluttered backgrounds under varying lighting conditions. This is made possible by fine-tuning data generation and domain randomization techniques and optimizing the inference pipeline, overcoming the generalization and performance shortcomings of the original PVN3D algorithm. Finally, we make the code, synthetic dataset, and all the pre-trained models available on GitHub.

Functional assessment of current upper limb prostheses: An integrated clinical and technological perspective.

Although recent technological developments in the field of bionic upper limb prostheses, their rejection rate remains excessively high. The reasons are diverse (e.g. lack of functionality, control complexity, and comfortability) and most of these are reported only through self-rated questionnaires. Indeed, there is no quantitative evaluation of the extent to which a novel prosthetic solution can effectively address users' needs compared to other technologies. This manuscript discusses the challenges and limitations of current upper limb prosthetic devices and evaluates their functionality through a standard functional assessment, the Assessment of Capacity for Myoelectric Control (ACMC). To include a good representation of technologies, the authors collect information from participants in the Cybathlon Powered Arm Prostheses Race 2016 and 2020. The article analyzes 7 hour and 41 min of video footage to evaluate the performance of different prosthetic devices in various tasks inspired by activities of daily living (ADL). The results show that commercially-available rigid hands perform well in dexterous grasping, while body-powered solutions are more reliable and convenient for competitive environments. The article also highlights the importance of wrist design and control modality for successful execution of ADL. Moreover, we discuss the limitations of the evaluation methodology and suggest improvements for future assessments. With regard to future development, this work highlights the need for research in intuitive control of multiple degrees of freedom, adaptive solutions, and the integration of sensory feedback.

The SoftHand Pro platform: a flexible prosthesis with a user-centered approach.

BACKGROUND: Among commercially-available upper-limb prostheses, the two most often used solutions are simple hook-style grippers and poly-articulated hands, which present a higher number of articulations and show a closer resemblance to biological limbs. In their majority, the former type of prostheses is body-powered, while the second type is controlled by myoelectric signals. Body-powered grippers are easy to control and allow a simple form of force feedback, frequently appreciated by users. However, they present limited versatility. Poly-articulated hands afford a wide range of grasp and manipulation types, but require enough residual muscle activation for dexterous control. Several factors, e.g. level of limb loss, personal preferences, cost, current occupation, and hobbies can influence the preference for one option over the other, and is always a result of the trade-off between system performance and users' needs. METHODS: The SoftHand Pro (SHP) is an artificial hand platform that has 19 independent joints (degrees-of-freedom), but is controlled by a single input. The design of this prosthesis is inspired by the concept of postural synergies in motor control and implemented with soft-robotic technologies. Their combination provides increased robustness, safe interaction and the execution of diverse grasps. The potential of the SHP is fully unleashed when users learn how to exploit its features and create an intimate relationship between the technical aspects of the prosthesis design and its control by the user. RESULTS: The great versatility of the SoftHand Pro (a reasearch protpotype) permitted its adaptation to the user requirements. This was experienced by the SoftHand Pro Team during the preparation for different CYBATHLON events (from 2016 to 2020). The mixed power and dexterous hand operations required by each task of the race is inspired by everyday tasks. Our prosthesis was driven by different pilots, with different habits and backgrounds. Consequently, the hand control modality was customized according to the user's preferences. Furthermore, the CYBATHLON tasks had some variations in this period, promoting the continuous development of our technology with a user-centered approach. In this paper, we describe the participation and preparation of the SoftHand Pro Team from 2016 to 2020 with three pilots and two different activation modalities, hybrid body-controlled and myoelectric control. CONCLUSIONS: We introduced our pilots, the implementation of the two control modalities, and describe the successful participation in all CYBATHLON events. This work proves the versatility of the system towards the user's preferences and the changes in the race requirements. Finally, we discussed how the CYBATHLON experience and the training in the real-world scenario have driven the evolution of our system and gave us remarkable insights for future perspectives.

Comparison between rigid and soft poly-articulated prosthetic hands in non-expert myo-electric users shows advantages of soft robotics.

Notwithstanding the advancement of modern bionic hands and the large variety of prosthetic hands in the market, commercial devices still present limited acceptance and percentage of daily use. While commercial prostheses present rigid mechanical structures, emerging trends in the design of robotic hands are moving towards soft technologies. Although this approach is inspired by nature and could be promising for prosthetic applications, there is scant literature concerning its benefits for end-users and in real-life scenarios. In this work, we evaluate and assess the role and the benefits of soft robotic technologies in the field of prosthetics. We propose a thorough comparison between rigid and soft characteristics of two poly-articulated hands in 5 non-expert myo-electric prosthesis users in pre- and post-therapeutic training conditions. The protocol includes two standard functional assessments, three surveys for user-perception, and three customized tests to evaluate the sense of embodiment. Results highlight that rigid hands provide a more precise grasp, while soft properties show higher functionalities thanks to their adaptability to different requirements, intuitive use and more natural execution of activities of daily living. This comprehensive evaluation suggests that softness could also promote a quick integration of the system in non-expert users.

An Adaptive Multi-Modal Control Strategy to Attenuate the Limb Position Effect in Myoelectric Pattern Recognition.

Over the last few decades, pattern recognition algorithms have shown promising results in the field of upper limb prostheses myoelectric control and are now gradually being incorporated in commercial devices. A widely used approach is based on a classifier which assigns a specific input value to a selected hand motion. While this method guarantees good performance and robustness within each class, it still shows limitations in adapting to different conditions encountered in real-world applications, such as changes in limb position or external loads. This paper proposes an adaptive method based on a pattern recognition classifier that takes advantage of an augmented dataset-i.e., representing variations in limb position or external loads-to selectively adapt to underrepresented variations. The proposed method was evaluated using a series of target achievement control tests with ten able-bodied volunteers. Results indicated a higher median completion rate >3.33% for the adapted algorithm compared to a classical pattern recognition classifier used as a baseline model. Subject-specific performance showed the potential for improved control after adaptation and a ≤13% completion rate; and in many instances, the adapted points were able to provide new information within classes. These preliminary results show the potential of the proposed method and encourage further development.

Usability Assessment of Body Controlled Electric Hand Prostheses: A Pilot Study.

Poly-articulated hands, actuated by multiple motors and controlled by surface myoelectric technologies, represent the most advanced aids among commercial prostheses. However, simple hook-like body-powered solutions are still preferred for their robustness and control reliability, especially for challenging environments (such as those encountered in manual work or developing countries). This study presents the mechatronic implementation and the usability assessment of the SoftHand Pro-Hybrid, a family of poly-articulated, electrically-actuated, and body-controlled artificial hands, which combines the main advantages of both body-powered and myoelectric systems in a single device. An assessment of the proposed system is performed with individuals with and without limb loss, using as a benchmark the SoftHand Pro, which shares the same soft mechanical architecture, but is controlled using surface electromyographic sensors. Results indicate comparable task performance between the two control methods and suggest the potential of the SoftHand Pro-Hybrid configurations as a viable alternative to myoelectric control, especially in work and demanding environments.

Exploring augmented grasping capabilities in a multi-synergistic soft bionic hand.

BACKGROUND: State-of-the-art bionic hands incorporate hi-tech devices which try to overcome limitations of conventional single grip systems. Unfortunately, their complexity often limits mechanical robustness and intuitive prosthesis control. Recently, the translation of neuroscientific theories (i.e. postural synergies) in software and hardware architecture of artificial devices is opening new approaches for the design and control of upper-limb prostheses. METHODS: Following these emerging principles, previous research on the SoftHand Pro, which embeds one physical synergy, showed promising results in terms of intuitiveness, robustness, and grasping performance. To explore these principles also in hands with augmented capabilities, this paper describes the SoftHand 2 Pro, a second generation of the device with 19 degrees-of-freedom and a second synergistic layer. After a description of the proposed device, the work explores a continuous switching control method based on a myoelectric pattern recognition classifier. RESULTS: The combined system was validated using standardized assessments with able-bodied and, for the first time, amputee subjects. Results show an average improvement of more than 30% of fine grasp capabilities and about 10% of hand function compared with the first generation SoftHand Pro. CONCLUSIONS: Encouraging results suggest how this approach could be a viable way towards the design of more natural, reliable, and intuitive dexterous hands.

Evaluation of a Simultaneous Myoelectric Control Strategy for a Multi-DoF Transradial Prosthesis.

While natural movements result from fluid coordination of multiple joints, commercial upper-limb prostheses are still limited to sequential control of multiple degrees of freedom (DoFs), or constrained to move along predefined patterns. To control multiple DoFs simultaneously, a probability-weighted regression (PWR) method has been proposed and has previously shown good performance with intramuscular electromyographic (EMG) sensors. This study aims to evaluate the PWR method for the simultaneous and proportional control of multiple DoFs using surface EMG sensors and compare the performance with a classical direct control strategy. To extract the maximum number of DoFs manageable by a user, a first analysis was conducted in a virtually simulated environment with eight able-bodied and four amputee subjects. Results show that, while using surface EMG degraded the PWR performance for the 3-DoFs control, the algorithm demonstrated excellent achievements in the 2-DoFs case. Finally, the two methods were compared on a physical experiment with amputee subjects using a hand-wrist prosthesis composed of the SoftHand Pro and the RIC Wrist Flexor. Results show comparable outcomes between the two controllers but a significantly higher wrist activation time for the PWR method, suggesting this novel method as a viable direction towards a more natural control of multi-DoFs.

Exploring Stiffness Modulation in Prosthetic Hands and Its Perceived Function in Manipulation and Social Interaction.

To physically interact with a rich variety of environments and to match situation-dependent requirements, humans adapt both the force and stiffness of their limbs. Reflecting this behavior in prostheses may promote a more natural and intuitive control and, consequently, improve prostheses acceptance in everyday life. This pilot study proposes a method to control a prosthetic robot hand and its impedance, and explores the utility of variable stiffness when performing activities of daily living and physical social interactions. The proposed method is capable of a simultaneous and proportional decoding of position and stiffness intentions from two surface electro-myographic sensors placed over a pair of antagonistic muscles. The feasibility of our approach is validated and compared to existing control modalities in a preliminary study involving one prosthesis user. The algorithm is implemented in a soft under-actuated prosthetic hand (SoftHand Pro). Then, we evaluate the usability of the proposed approach while executing a variety of tasks. Among these tasks, the user interacts with other 12 able-bodied subjects, whose experiences were also assessed. Several statistically significant aspects from the System Usability Scale indicate user's preference of variable stiffness control over low or high constant stiffness due to its reactivity and adaptability. Feedback reported by able-bodied subjects reveal a general tendency to favor soft interaction, i.e., low stiffness, which is perceived more human-like and comfortable. These combined results suggest the use of variable stiffness as a viable compromise between firm control and safe interaction which is worth investigating further.

An Objective Functional Evaluation of Myoelectrically-Controlled Hand Prostheses: A Pilot Study Using the Virtual Peg Insertion Test.

Assessing upper limb prostheses and their influence when performing goal-directed activities is essential to compare the quality of different devices and optimize their control settings. Currently available assessments are often subjective, insensitive, and cannot provide a detailed evaluation of prostheses and their usage. The goal of this pilot study was to explore the feasibility of using the Virtual Peg Insertion Test (VPIT) to provide an in-depth assessment of a prosthesis and its functional performance. One transradial amputee performed the goal-directed manipulation task of the VPIT with the sound body side and four different myoelectrically-controlled prostheses. The subject was able to complete the VPIT protocol successfully with technically advanced prosthesis (two out of four devices). The kinematic- and kinetic-based objective evaluation measures extracted from the VPIT were able to capture clear differences between the sound and amputated body side and were able to identify varying movement patterns for different prostheses. Additionally, the outcome measures were sensitive to changes in prosthesis control settings and showed clear trends across measures of subjectively perceived prosthesis quality assessed through a questionnaire. This work demonstrates the general feasibility of objectively evaluating functional prosthesis usage with the VPIT.

Design and Assessment of Control Maps for Multi-Channel sEMG-Driven Prostheses and Supernumerary Limbs.

Proportional and simultaneous control algorithms are considered as one of the most effective ways of mapping electromyographic signals to an artificial device. However, the applicability of these methods is limited by the high number of electromyographic features that they require to operate-typically twice as many the actuators to be controlled. Indeed, extracting many independent electromyographic signals is challenging for a number of reasons-ranging from technological to anatomical. On the contrary, the number of actively moving parts in classic prostheses or extra-limbs is often high. This paper faces this issue, by proposing and experimentally assessing a set of algorithms which are capable of proportionally and simultaneously control as many actuators as there are independent electromyographic signals available. Two sets of solutions are considered. The first uses as input electromyographic signals only, while the second adds postural measurements to the sources of information. At first, all the proposed algorithms are experimentally tested in terms of precision, efficiency, and usability on twelve able-bodied subjects, in a virtual environment. A state-of-the-art controller using twice the amount of electromyographic signals as input is adopted as benchmark. We then performed qualitative tests, where the maps are used to control a prototype of upper limb prosthesis. The device is composed of a robotic hand and a wrist implementing active prono-supination movement. Eight able-bodied subjects participated to this second round of testings. Finally, the proposed strategies were tested in exploratory experiments involving two subjects with limb loss. Results coming from the evaluations in virtual and realistic settings show encouraging results and suggest the effectiveness of the proposed approach.

SoftHand at the CYBATHLON: a user's experience.

BACKGROUND: Roughly one-quarter of upper limb prosthesis users reject their prosthesis. Reasons for rejection range from comfort, to cost, aesthetics, function, and more. This paper follows a single user from training with and testing of a novel upper-limb myoelectric prosthesis (the SoftHand Pro) for participation in the CYBATHLON rehearsal to training for and competing in the CYBATHLON 2016 with a figure-of-nine harness controlled powered prosthesis (SoftHand Pro-H) to explore the feasibility and usability of a flexible anthropomorphic prosthetic hand. METHODS: The CYBATHLON pilot took part in multiple in-lab training sessions with the SoftHand Pro and SoftHand Pro-H; these sessions focused on basic control and use of the prosthetic devices and direct training of the tasks in the CYBATHLON. He used these devices in competition in the Powered Arm Prosthesis Race in the CYBATHLON rehearsal and 2016 events. RESULTS: In training for the CYBATHLON rehearsal, the subject was able to quickly improve performance with the myoelectric SHP despite typically using a body-powered prosthetic hook. The subject improved further with additional training using the figure-of-nine harness-controlled SHPH in preparation for the CYBATHLON. The Pilot placed 3rd (out of 4) in the rehearsal. In the CYBATHLON, he placed 5th (out of 12) and was one of only two pilots who successfully completed all tasks in the competition, having the second-highest score overall. CONCLUSIONS: Results with the SoftHand Pro and Pro-H suggest it to be a viable alternative to existing anthropomorphic hands and show that the unique flexibility of the hand is easily learned and exploited.

Preliminary results toward a naturally controlled multi-synergistic prosthetic hand.

Robotic hands embedding human motor control principles in their mechanical design are getting increasing interest thanks to their simplicity and robustness, combined with good performance. Another key aspect of these hands is that humans can use them very effectively thanks to the similarity of their behavior with real hands. Nevertheless, controlling more than one degree of actuation remains a challenging task. In this paper, we take advantage of these characteristics in a multi-synergistic prosthesis. We propose an integrated setup composed of Pisa/IIT SoftHand 2 and a control strategy which simultaneously and proportionally maps the human hand movements to the robotic hand. The control technique is based on a combination of non-negative matrix factorization and linear regression algorithms. It also features a real-time continuous posture compensation of the electromyographic signals based on an IMU. The algorithm is tested on five healthy subjects through an experiment in a virtual environment. In a separate experiment, the efficacy of the posture compensation strategy is evaluated on five healthy subjects and, finally, the whole setup is successfully tested in performing realistic daily life activities.