Embodied bidirectional simulation of a spiking cortico-basal ganglia-cerebellar-thalamic brain model and a mouse musculoskeletal body model distributed across computers including the supercomputer Fugaku.

Embodied simulation with a digital brain model and a realistic musculoskeletal body model provides a means to understand animal behavior and behavioral change. Such simulation can be too large and complex to conduct on a single computer, and so distributed simulation across multiple computers over the Internet is necessary. In this study, we report our joint effort on developing a spiking brain model and a mouse body model, connecting over the Internet, and conducting bidirectional simulation while synchronizing them. Specifically, the brain model consisted of multiple regions including secondary motor cortex, primary motor and somatosensory cortices, basal ganglia, cerebellum and thalamus, whereas the mouse body model, provided by the Neurorobotics Platform of the Human Brain Project, had a movable forelimb with three joints and six antagonistic muscles to act in a virtual environment. Those were simulated in a distributed manner across multiple computers including the supercomputer Fugaku, which is the flagship supercomputer in Japan, while communicating via Robot Operating System (ROS). To incorporate models written in C/C++ in the distributed simulation, we developed a C++ version of the rosbridge library from scratch, which has been released under an open source license. These results provide necessary tools for distributed embodied simulation, and demonstrate its possibility and usefulness toward understanding animal behavior and behavioral change.

Network Layer Analysis for a RL-Based Robotic Reaching Task.

Recent experiments indicate that pretraining of end-to-end reinforcement learning neural networks on general tasks can speed up the training process for specific robotic applications. However, it remains open if these networks form general feature extractors and a hierarchical organization that can be reused as in, for example, convolutional neural networks. In this study, we analyze the intrinsic neuron activation in networks trained for target reaching of robot manipulators with increasing joint number and analyze the individual neuron activation distribution within the network. We introduce a pruning algorithm to increase network information density and depict correlations of neuron activation patterns. Finally, we search for projections of neuron activation among networks trained for robot kinematics of different complexity. As a result, we show that the input and output network layers entail more distinct neuron activation in contrast to inner layers. Our pruning algorithm reduces the network size significantly and increases the distance of neuron activation while keeping a high performance in training and evaluation. Our results demonstrate that robots with small difference in joint number show higher layer-wise projection accuracy, whereas more distinct robot kinematics reveal dominant projections to the first layer.

Evaluating Muscle Synergies With EMG Data and Physics Simulation in the Neurorobotics Platform.

Although we can measure muscle activity and analyze their activation patterns, we understand little about how individual muscles affect the joint torque generated. It is known that they are controlled by circuits in the spinal cord, a system much less well-understood than the cortex. Knowing the contribution of the muscles toward a joint torque would improve our understanding of human limb control. We present a novel framework to examine the control of biomechanics using physics simulations informed by electromyography (EMG) data. These signals drive a virtual musculoskeletal model in the Neurorobotics Platform (NRP), which we then use to evaluate resulting joint torques. We use our framework to analyze raw EMG data collected during an isometric knee extension study to identify synergies that drive a musculoskeletal lower limb model. The resulting knee torques are used as a reference for genetic algorithms (GA) to generate new simulated activation patterns. On the platform the GA finds solutions that generate torques matching those observed. Possible solutions include synergies that are similar to those extracted from the human study. In addition, the GA finds activation patterns that are different from the biological ones while still producing the same knee torque. The NRP forms a highly modular integrated simulation platform allowing these in silico experiments. We argue that our framework allows for research of the neurobiomechanical control of muscles during tasks, which would otherwise not be possible.

Deploying and Optimizing Embodied Simulations of Large-Scale Spiking Neural Networks on HPC Infrastructure.

Simulating the brain-body-environment trinity in closed loop is an attractive proposal to investigate how perception, motor activity and interactions with the environment shape brain activity, and vice versa. The relevance of this embodied approach, however, hinges entirely on the modeled complexity of the various simulated phenomena. In this article, we introduce a software framework that is capable of simulating large-scale, biologically realistic networks of spiking neurons embodied in a biomechanically accurate musculoskeletal system that interacts with a physically realistic virtual environment. We deploy this framework on the high performance computing resources of the EBRAINS research infrastructure and we investigate the scaling performance by distributing computation across an increasing number of interconnected compute nodes. Our architecture is based on requested compute nodes as well as persistent virtual machines; this provides a high-performance simulation environment that is accessible to multi-domain users without expert knowledge, with a view to enable users to instantiate and control simulations at custom scale via a web-based graphical user interface. Our simulation environment, entirely open source, is based on the Neurorobotics Platform developed in the context of the Human Brain Project, and the NEST simulator. We characterize the capabilities of our parallelized architecture for large-scale embodied brain simulations through two benchmark experiments, by investigating the effects of scaling compute resources on performance defined in terms of experiment runtime, brain instantiation and simulation time. The first benchmark is based on a large-scale balanced network, while the second one is a multi-region embodied brain simulation consisting of more than a million neurons and a billion synapses. Both benchmarks clearly show how scaling compute resources improves the aforementioned performance metrics in a near-linear fashion. The second benchmark in particular is indicative of both the potential and limitations of a highly distributed simulation in terms of a trade-off between computation speed and resource cost. Our simulation architecture is being prepared to be accessible for everyone as an EBRAINS service, thereby offering a community-wide tool with a unique workflow that should provide momentum to the investigation of closed-loop embodiment within the computational neuroscience community.

The Neurorobotics Platform Robot Designer: Modeling Morphologies for Embodied Learning Experiments.

The more we investigate the principles of motion learning in biological systems, the more we reveal the central role that body morphology plays in motion execution. Not only does anatomy define the kinematics and therefore the complexity of possible movements, but it now becomes clear that part of the computation required for motion control is offloaded to body dynamics (a phenomenon referred to as "Morphological Computation.") Consequentially, a proper design of body morphology is essential to carry out meaningful simulations on motor control of robotic and musculoskeletal systems. The design should not be fixed for simulation experiments beforehand, but is a central research aspect in every motion learning experiment that requires continuous adaptation during the experimental phase. We herein introduce a plugin for the 3D modeling suite Blender that enables researchers to design morphologies for simulation experiments in, particularly but not restricted to, the Neurorobotics Platform. We include design capabilities for both musculoskeletal bodies, as well as robotic systems in the Robot Designer. Thereby, we hope to not only foster understanding of biological motions and enabling better robot designs, but enabling true Neurorobotic experiments that may consist of biomimetic models such as tendon-driven robot as a mix of both or a transition between both biology and technology. This plugin helps researchers design and parameterize models with a Graphical User Interface and thus simplifies and speeds up the overall design process.

Adapting Highly-Dynamic Compliant Movements to Changing Environments: A Benchmark Comparison of Reflex- vs. CPG-Based Control Strategies.

To control highly-dynamic compliant motions such as running or hopping, vertebrates rely on reflexes and Central Pattern Generators (CPGs) as core strategies. However, decoding how much each strategy contributes to the control and how they are adjusted under different conditions is still a major challenge. To help solve this question, the present paper provides a comprehensive comparison of reflexes, CPGs and a commonly used combination of the two applied to a biomimetic robot. It leverages recent findings indicating that in mammals both control principles act within a low-dimensional control submanifold. This substantially reduces the search space of parameters and enables the quantifiable comparison of the different control strategies. The chosen metrics are motion stability and energy efficiency, both key aspects for the evolution of the central nervous system. We find that neither for stability nor energy efficiency it is favorable to apply the state-of-the-art approach of a continuously feedback-adapted CPG. In both aspects, a pure reflex is more effective, but the pure CPG allows easy signal alteration when needed. Additionally, the hardware experiments clearly show that the shape of a control signal has a strong influence on energy efficiency, while previous research usually only focused on frequency alignment. Both findings suggest that currently used methods to combine the advantages of reflexes and CPGs can be improved. In future research, possible combinations of the control strategies should be reconsidered, specifically including the modulation of the control signal's shape. For this endeavor, the presented setup provides a valuable benchmark framework to enable the quantitative comparison of different bioinspired control principles.

Hebbian learning for online prediction, neural recall and classical conditioning of anthropomimetic robot arm motions.

Classical conditioning plays a vital role in learning in every mammal. It is based on unsupervised neural learning embodied in a physical body that is in continuous interaction with the environment. Embedding the hierarchical temporal memory (HTM) in the closed loop of the sensorimotor space of a Myorobotics tendon-driven robotic arm we demonstrate learning, prediction and control of biomimetic body motions. Experiments finally lead to conditioned reactions in natural interaction with a human partner. The HTM is able to learn arm movements generated by interaction with a human partner in a short time. It predicts future positions in different time scales up to seconds in advance. Closing the loop we utilize HTM predictions for motor control. Hereby learned motions are recalled from synaptic connections proactively continuing motion execution. Association, prediction and control are required by the HTM for conditioning according to Pavlov: neutral stimuli get associated with motions, and after learning sensor impulses can trigger single arm lifting motions. Hereby, both the motions and the stimuli are learned from the environment and get associated efficiently. We can demonstrate high biological plausibility as for example even input variations result in similar variations in the action output. The robotic system consisting of biologically-derived hardware and software components utilizes only unsupervised Hebbian learning to act autonomously. Learning is executed in real time, can handle natural variations of human motions and takes morphologically plausible sensor input into account. The setup is fully scalable due to its modularity. Hereby, novel applications for the HTM are opened: it can be used in musculoskeletal robot control scenarios and robots being able to interactively learn from human partners and the environment.