Synergy quality assessment of muscle modules for determining learning performance using a realistic musculoskeletal model.

How our central nervous system efficiently controls our complex musculoskeletal system is still debated. The muscle synergy hypothesis is proposed to simplify this complex system by assuming the existence of functional neural modules that coordinate several muscles. Modularity based on muscle synergies can facilitate motor learning without compromising task performance. However, the effectiveness of modularity in motor control remains debated. This ambiguity can, in part, stem from overlooking that the performance of modularity depends on the mechanical aspects of modules of interest, such as the torque the modules exert. To address this issue, this study introduces two criteria to evaluate the quality of module sets based on commonly used performance metrics in motor learning studies: the accuracy of torque production and learning speed. One evaluates the regularity in the direction of mechanical torque the modules exert, while the other evaluates the evenness of its magnitude. For verification of our criteria, we simulated motor learning of torque production tasks in a realistic musculoskeletal system of the upper arm using feed-forward neural networks while changing the control conditions. We found that the proposed criteria successfully explain the tendency of learning performance in various control conditions. These result suggest that regularity in the direction of and evenness in magnitude of mechanical torque of utilized modules are significant factor for determining learning performance. Although the criteria were originally conceived for an error-based learning scheme, the approach to pursue which set of modules is better for motor control can have significant implications in other studies of modularity in general.

Integrated Quantitative Evaluation of Spatial Cognition and Motor Function with HoloLens Mixed Reality.

The steady increase in the aging population worldwide is expected to cause a shortage of doctors and therapists for older people. This demographic shift requires more efficient and automated systems for rehabilitation and physical ability evaluations. Rehabilitation using mixed reality (MR) technology has attracted much attention in recent years. MR displays virtual objects on a head-mounted see-through display that overlies the user's field of vision and allows users to manipulate them as if they exist in reality. However, tasks in previous studies applying MR to rehabilitation have been limited to tasks in which the virtual objects are static and do not interact dynamically with the surrounding environment. Therefore, in this study, we developed an application to evaluate cognitive and motor functions with the aim of realizing a rehabilitation system that is dynamic and has interaction with the surrounding environment using MR technology. The developed application enabled effective evaluation of the user's spatial cognitive ability, task skillfulness, motor function, and decision-making ability. The results indicate the usefulness and feasibility of MR technology to quantify motor function and spatial cognition both for static and dynamic tasks in rehabilitation.

Latent Representation-Based Learning Controller for Pneumatic and Hydraulic Dual Actuation of Pressure-Driven Soft Actuators.

The pneumatic and hydraulic dual actuation of pressure-driven soft actuators (PSAs) is promising because of their potential to develop novel practical soft robots and expand the range of soft robot applications. However, the physical characteristics of air and water are largely different, which makes it challenging to quickly adapt to a selected actuation method and achieve method-independent accurate control performance. Herein, we propose a novel LAtent Representation-based Feedforward Neural Network (LAR-FNN) for dual actuation. The LAR-FNN consists of an autoencoder (AE) and a feedforward neural network (FNN). The AE generates a latent representation of a PSA from a 30-s stairstep response. Subsequently, the FNN provides an individual inverse model of the target PSA and calculates feedforward control input by using the latent representation. The experimental results with PSAs demonstrate that the LAR-FNN can meet the requirements of dual actuation control (i.e., accurate control performance regardless of the actuation method with a short adaptation time) with a single neural network. The results suggest that a LAR-FNN can contribute to soft dual-actuation robot development and the field of soft robotics.

Transhumeral Arm Reaching Motion Prediction through Deep Reinforcement Learning-Based Synthetic Motion Cloning.

The lack of intuitive controllability remains a primary challenge in enabling transhumeral amputees to control a prosthesis for arm reaching with residual limb kinematics. Recent advancements in prosthetic arm control have focused on leveraging the predictive capabilities of artificial neural networks (ANNs) to automate elbow joint motion and wrist pronation-supination during target reaching tasks. However, large quantities of human motion data collected from different subjects for various activities of daily living (ADL) tasks are required to train these ANNs. For example, the reaching motion can be altered when the height of the desk is changed; however, it is cumbersome to conduct human experiments for all conditions. This paper proposes a framework for cloning motion datasets using deep reinforcement learning (DRL) to cater to training data requirements. DRL algorithms have been demonstrated to create human-like synergistic motion in humanoid agents to handle redundancy and optimize movements. In our study, we collected real motion data from six individuals performing multi-directional arm reaching tasks in the horizontal plane. We generated synthetic motion data that mimicked similar arm reaching tasks by utilizing a physics simulation and DRL-based arm manipulation. We then trained a CNN-LSTM network with different configurations of training motion data, including DRL, real, and hybrid datasets, to test the efficacy of the cloned motion data. The results of our evaluation showcase the effectiveness of the cloned motion data in training the ANN to predict natural elbow motion accurately across multiple subjects. Furthermore, motion data augmentation through combining real and cloned motion datasets has demonstrated the enhanced robustness of the ANN by supplementing and diversifying the limited training data. These findings have significant implications for creating synthetic dataset resources for various arm movements and fostering strategies for automatized prosthetic elbow motion.

A Survey of Sim-to-Real Transfer Techniques Applied to Reinforcement Learning for Bioinspired Robots.

The state-of-the-art reinforcement learning (RL) techniques have made innumerable advancements in robot control, especially in combination with deep neural networks (DNNs), known as deep reinforcement learning (DRL). In this article, instead of reviewing the theoretical studies on RL, which were almost fully completed several decades ago, we summarize some state-of-the-art techniques added to commonly used RL frameworks for robot control. We mainly review bioinspired robots (BIRs) because they can learn to locomote or produce natural behaviors similar to animals and humans. With the ultimate goal of practical applications in real world, we further narrow our review scope to techniques that could aid in sim-to-real transfer. We categorized these techniques into four groups: 1) use of accurate simulators; 2) use of kinematic and dynamic models; 3) use of hierarchical and distributed controllers; and 4) use of demonstrations. The purposes of these four groups of techniques are to supply general and accurate environments for RL training, improve sampling efficiency, divide and conquer complex motion tasks and redundant robot structures, and acquire natural skills. We found that, by synthetically using these techniques, it is possible to deploy RL on physical BIRs in actuality.

Quantifying Motor and Cognitive Function of the Upper Limb Using Mixed Reality Smartglasses.

In our aging world, the need to measure and evaluate motor and cognitive functions and to automate physical and occupational therapy will increase in the future. Many studies on VR-based rehabilitation systems are already underway. However, there are some issues such as the risk of falling or crashing due to the complete blockage of visual information, VR sickness, and lack of reality. The purpose of this research is to develop a system that simultaneously measures and evaluates multiple abilities and functions, such as motor function, cognitive function, and prediction ability, by using mixed reality (MR) smartglasses technology that enables interaction with spatially arranged objects while maintaining real-world information. In this study, we focused on the motor function of the upper limbs and cognitive function, and measured finger and gaze movements during a reaching task. In addition, we developed a game-based task for occupational therapy in a MR environment and reported the results.

Systematic Motion Integration with Multiple Depth Cameras Allowing Sensor Movement for Stable Skeleton Tracking.

In recent years, markerless motion capture using a depth camera or RGB camera without any restriction on the subject has been attracting attention. Especially, depth cameras such as Kinect and RealSense allow instantaneous motion capture even at home outside lab environment, which is attractive for rehabilitation usage. However, single depth camera can capture steadily skeleton only when the subject stands facing to camera for the limited range, thus it is hard to apply to track skeletons while walking. Multiple depth cameras setting may allow to expand the range, but it can involve non-practical calibration process and can affect instantaneous capture advantage of depth camera. In this study, we propose a systematic method to integrate the motion information of skeletal models obtained from multiple depth cameras. The proposed method can perform a quick calibration using skeletal models instead of external reference objects, and estimate the spatial relationship of the sensors that allows the depth camera to move. The result demonstrates stable skeleton tracking free from occlusion problem keeping instantaneous capture capability of depth cameras.

Game-based Evaluation of Whole-body Movement Functions with CoM Stability and Motion Smoothness.

In the field of rehabilitation, there is a great demand for an automatic and quantitative evaluation system. The balance ability is an essential factor for motor function evaluation related to posture control. Although balance ability is assessed using various indices in current clinical situations, most of previous studies developing an automatic evaluation system have used only a single particular index for balance evaluation. In this study, we developed a system that evaluates whole-body motor function using multiple indices based on the trajectory of the center of mass (CoM) and the motion smoothness. The system is inexpensive and little physical burden because the evaluation indices are calculated from the skeleton tracked by Kinect in a game environment. We attempt to capture the differences in individual motor functions which are difficult to be detected by qualitative visual observation.

Motor synergy generalization framework for new targets in multi-planar and multi-directional reaching task.

Humans can rapidly adapt to new situations, even though they have redundant degrees of freedom (d.f.). Previous studies in neuroscience revealed that human movements could be accounted for by low-dimensional control signals, known as motor synergies. Many studies have suggested that humans use the same repertories of motor synergies among similar tasks. However, it has not yet been confirmed whether the combinations of motor synergy repertories can be re-used for new targets in a systematic way. Here we show that the combination of motor synergies can be generalized to new targets that each repertory cannot handle. We use the multi-directional reaching task as an example. We first trained multiple policies with limited ranges of targets by reinforcement learning and extracted sets of motor synergies. Finally, we optimized the activation patterns of sets of motor synergies and demonstrated that combined motor synergy repertories were able to reach new targets that were not achieved with either original policies or single repertories of motor synergies. We believe this is the first study that has succeeded in motor synergy generalization for new targets in new planes, using a full 7-d.f. arm model, which is a realistic mechanical environment for general reaching tasks.

Multimodal bipedal locomotion generation with passive dynamics via deep reinforcement learning.

Generating multimodal locomotion in underactuated bipedal robots requires control solutions that can facilitate motion patterns for drastically different dynamical modes, which is an extremely challenging problem in locomotion-learning tasks. Also, in such multimodal locomotion, utilizing body morphology is important because it leads to energy-efficient locomotion. This study provides a framework that reproduces multimodal bipedal locomotion using passive dynamics through deep reinforcement learning (DRL). An underactuated bipedal model was developed based on a passive walker, and a controller was designed using DRL. By carefully planning the weight parameter settings of the DRL reward function during the learning process based on a curriculum learning method, the bipedal model successfully learned to walk, run, and perform gait transitions by adjusting only one command input. These results indicate that DRL can be applied to generate various gaits with the effective use of passive dynamics.

Individual deformability compensation of soft hydraulic actuators through iterative learning-based neural network.

Robotic devices with soft actuators have been developed to realize the effective rehabilitation of patients with motor paralysis by enabling soft and safe interaction. However, the control of such robots is challenging, especially owing to the difference in the individual deformability occurring in manual fabrication of soft actuators. Furthermore, soft actuators used in wearable rehabilitation devices involve a large response delay which hinders the application of such devices for at-home rehabilitation. In this paper, a feed-forward control method for soft actuators with a large response delay, comprising a simple feed-forward neural network (FNN) and an iterative learning controller (ILC), is proposed. The proposed method facilitates the effective learning and acquisition of the inverse model (i.e. the model that can generate control input to the soft actuator from a target trajectory) of soft actuators. First, the ILC controls a soft actuator and iteratively learns the actuator deformability. Subsequently, the FNN is trained to obtain the inverse model of the soft actuator. The control results of the ILC are used as training datasets for supervised learning of the FNN to ensure that it can efficiently acquire the inverse model of the soft actuator, including the deformability and the response delay. Experiments with fiber-reinforced soft bending hydraulic actuators are conducted to evaluate the proposed method. The results show that the ILC can learn and compensate for the actuator deformability. Moreover, the iterative learning-based FNN serves to achieve a precise tracking performance on various generalized trajectories. These facts suggest that the proposed method can contribute to the development of robotic rehabilitation devices with soft actuators and the field of soft robotics.