neuroWalknet, a controller for hexapod walking allowing for context dependent behavior.

Decentralized control has been established as a key control principle in insect walking and has been successfully leveraged to account for a wide range of walking behaviors in the proposed neuroWalknet architecture. This controller allows for walking patterns at different velocities in both, forward and backward direction-quite similar to the behavior shown in stick insects-, for negotiation of curves, and for robustly dealing with various disturbances. While these simulations focus on the cooperation of different, decentrally controlled legs, here we consider a set of biological experiments not yet been tested by neuroWalknet, that focus on the function of the individual leg and are context dependent. These intraleg studies deal with four groups of interjoint reflexes. The reflexes are elicited by stimulation of the femoral chordotonal organ (fCO) or groups of campaniform sensilla (CS). Motor output signals are recorded from the alpha-joint, the beta-joint or the gamma-joint of the leg. Furthermore, the influence of these sensory inputs to artificially induced oscillations by application of pilocarpine has been studied. Although these biological data represent results obtained from different local reflexes in different contexts, they fit with and are embedded into the behavior shown by the global structure of neuroWalknet. In particular, a specific and intensively studied behavior, active reaction, has since long been assumed to represent a separate behavioral element, from which it is not clear why it occurs in some situations, but not in others. This question could now be explained as an emergent property of the holistic structure of neuroWalknet which has shown to be able to produce artificially elicited pilocarpine-driven oscillation that can be controlled by sensory input without the need of explicit innate CPG structures. As the simulation data result from a holistic system, further results were obtained that could be used as predictions to be tested in further biological experiments.

Decentralized control and local information for robust and adaptive decentralized Deep Reinforcement Learning.

Decentralization is a central characteristic of biological motor control that allows for fast responses relying on local sensory information. In contrast, the current trend of Deep Reinforcement Learning (DRL) based approaches to motor control follows a centralized paradigm using a single, holistic controller that has to untangle the whole input information space. This motivates to ask whether decentralization as seen in biological control architectures might also be beneficial for embodied sensori-motor control systems when using DRL. To answer this question, we provide an analysis and comparison of eight control architectures for adaptive locomotion that were derived for a four-legged agent, but with their degree of decentralization varying systematically between the extremes of fully centralized and fully decentralized. Our comparison shows that learning speed is significantly enhanced in distributed architectures-while still reaching the same high performance level of centralized architectures-due to smaller search spaces and local costs providing more focused information for learning. Second, we find an increased robustness of the learning process in the decentralized cases-it is less demanding to hyperparameter selection and less prone to becoming trapped in poor local minima. Finally, when examining generalization to uneven terrains-not used during training-we find best performance for an intermediate architecture that is decentralized, but integrates only local information from both neighboring legs. Together, these findings demonstrate beneficial effects of distributing control into decentralized units and relying on local information. This appears as a promising approach towards more robust DRL and better generalization towards adaptive behavior.

Simulation across representation: The interplay of schemas and simulation-based inference on different levels of abstraction.

Language comprehension of action verbs recruits embodied representations in the brain that are assumed to invoke a mental simulation (e.g., "grasping a peanut"). This extends to abstract concepts, as well ("grasping an idea"). We, therefore, argue that mental simulation works across levels of abstractness and involves higher-level schematic structures that subsume a generic structure of actions and events.

Decentralized control of insect walking: A simple neural network explains a wide range of behavioral and neurophysiological results.

Controlling the six legs of an insect walking in an unpredictable environment is a challenging task, as many degrees of freedom have to be coordinated. Solutions proposed to deal with this task are usually based on the highly influential concept that (sensory-modulated) central pattern generators (CPG) are required to control the rhythmic movements of walking legs. Here, we investigate a different view. To this end, we introduce a sensor based controller operating on artificial neurons, being applied to a (simulated) insectoid robot required to exploit the "loop through the world" allowing for simplification of neural computation. We show that such a decentralized solution leads to adaptive behavior when facing uncertain environments which we demonstrate for a broad range of behaviors never dealt with in a single system by earlier approaches. This includes the ability to produce footfall patterns such as velocity dependent "tripod", "tetrapod", "pentapod" as well as various stable intermediate patterns as observed in stick insects and in Drosophila. These patterns are found to be stable against disturbances and when starting from various leg configurations. Our neuronal architecture easily allows for starting or interrupting a walk, all being difficult for CPG controlled solutions. Furthermore, negotiation of curves and walking on a treadmill with various treatments of individual legs is possible as well as backward walking and performing short steps. This approach can as well account for the neurophysiological results usually interpreted to support the idea that CPGs form the basis of walking, although our approach is not relying on explicit CPG-like structures. Application of CPGs may however be required for very fast walking. Our neuronal structure allows to pinpoint specific neurons known from various insect studies. Interestingly, specific common properties observed in both insects and crustaceans suggest a significance of our controller beyond the realm of insects.

Deep Learning for Understanding Satellite Imagery: An Experimental Survey.

Translating satellite imagery into maps requires intensive effort and time, especially leading to inaccurate maps of the affected regions during disaster and conflict. The combination of availability of recent datasets and advances in computer vision made through deep learning paved the way toward automated satellite image translation. To facilitate research in this direction, we introduce the Satellite Imagery Competition using a modified SpaceNet dataset. Participants had to come up with different segmentation models to detect positions of buildings on satellite images. In this work, we present five approaches based on improvements of U-Net and Mask R-Convolutional Neuronal Networks models, coupled with unique training adaptations using boosting algorithms, morphological filter, Conditional Random Fields and custom losses. The good results-as high as A P = 0.937 and A R = 0.959 -from these models demonstrate the feasibility of Deep Learning in automated satellite image annotation.

Integrative Biomimetics of Autonomous Hexapedal Locomotion.

Despite substantial advances in many different fields of neurorobotics in general, and biomimetic robots in particular, a key challenge is the integration of concepts: to collate and combine research on disparate and conceptually disjunct research areas in the neurosciences and engineering sciences. We claim that the development of suitable robotic integration platforms is of particular relevance to make such integration of concepts work in practice. Here, we provide an example for a hexapod robotic integration platform for autonomous locomotion. In a sequence of six focus sections dealing with aspects of intelligent, embodied motor control in insects and multipedal robots-ranging from compliant actuation, distributed proprioception and control of multiple legs, the formation of internal representations to the use of an internal body model-we introduce the walking robot HECTOR as a research platform for integrative biomimetics of hexapedal locomotion. Owing to its 18 highly sensorized, compliant actuators, light-weight exoskeleton, distributed and expandable hardware architecture, and an appropriate dynamic simulation framework, HECTOR offers many opportunities to integrate research effort across biomimetics research on actuation, sensory-motor feedback, inter-leg coordination, and cognitive abilities such as motion planning and learning of its own body size.

Transfer of Spatial Contact Information Among Limbs and the Notion of Peripersonal Space in Insects.

Internal representation of far-range space in insects is well established, as it is necessary for navigation behavior. Although it is likely that insects also have an internal representation of near-range space, the behavioral evidence for the latter is much less evident. Here, we estimate the size and shape of the spatial equivalent of a near-range representation that is constituted by somatosensory sampling events. To do so, we use a large set of experimental whole-body motion capture data on unrestrained walking, climbing and searching behavior in stick insects of the species Carausius morosus to delineate 'action volumes' and 'contact volumes' for both antennae and all six legs. As these volumes are derived from recorded sampling events, they comprise a volume equivalent to a representation of coinciding somatosensory and motor activity. Accordingly, we define this volume as the peripersonal space of an insect. It is of immediate behavioral relevance, because it comprises all potential external object locations within the action range of the body. In a next step, we introduce the notion of an affordance space as that part of peripersonal space within which contact-induced spatial estimates lie within the action ranges of more than one limb. Because the action volumes of limbs overlap in this affordance space, spatial information from one limb can be used to control the movement of another limb. Thus, it gives rise to an affordance as known for contact-induced reaching movements and spatial coordination of footfall patterns in stick insects. Finally, we probe the computational properties of the experimentally derived affordance space for pairs of neighboring legs. This is done by use of artificial neural networks that map the posture of one leg into a target posture of another leg with identical foot position.

ReaCog, a Minimal Cognitive Controller Based on Recruitment of Reactive Systems.

It has often been stated that for a neuronal system to become a cognitive one, it has to be large enough. In contrast, we argue that a basic property of a cognitive system, namely the ability to plan ahead, can already be fulfilled by small neuronal systems. As a proof of concept, we propose an artificial neural network, termed reaCog, that, first, is able to deal with a specific domain of behavior (six-legged-walking). Second, we show how a minor expansion of this system enables the system to plan ahead and deploy existing behavioral elements in novel contexts in order to solve current problems. To this end, the system invents new solutions that are not possible for the reactive network. Rather these solutions result from new combinations of given memory elements. This faculty does not rely on a dedicated system being more or less independent of the reactive basis, but results from exploitation of the reactive basis by recruiting the lower-level control structures in a way that motor planning becomes possible as an internal simulation relying on internal representation being grounded in embodied experiences.

A hexapod walker using a heterarchical architecture for action selection.

Moving in a cluttered environment with a six-legged walking machine that has additional body actuators, therefore controlling 22 DoFs, is not a trivial task. Already simple forward walking on a flat plane requires the system to select between different internal states. The orchestration of these states depends on walking velocity and on external disturbances. Such disturbances occur continuously, for example due to irregular up-and-down movements of the body or slipping of the legs, even on flat surfaces, in particular when negotiating tight curves. The number of possible states is further increased when the system is allowed to walk backward or when front legs are used as grippers and cannot contribute to walking. Further states are necessary for expansion that allow for navigation. Here we demonstrate a solution for the selection and sequencing of different (attractor) states required to control different behaviors as are forward walking at different speeds, backward walking, as well as negotiation of tight curves. This selection is made by a recurrent neural network (RNN) of motivation units, controlling a bank of decentralized memory elements in combination with the feedback through the environment. The underlying heterarchical architecture of the network allows to select various combinations of these elements. This modular approach representing an example of neural reuse of a limited number of procedures allows for adaptation to different internal and external conditions. A way is sketched as to how this approach may be expanded to form a cognitive system being able to plan ahead. This architecture is characterized by different types of modules being arranged in layers and columns, but the complete network can also be considered as a holistic system showing emergent properties which cannot be attributed to a specific module.

Walknet, a bio-inspired controller for hexapod walking.

Walknet comprises an artificial neural network that allows for the simulation of a considerable amount of behavioral data obtained from walking and standing stick insects. It has been tested by kinematic and dynamic simulations as well as on a number of six-legged robots. Over the years, various different expansions of this network have been provided leading to different versions of Walknet. This review summarizes the most important biological findings described by Walknet and how they can be simulated. Walknet shows how a number of properties observed in insects may emerge from a decentralized architecture. Examples are the continuum of so-called "gaits," coordination of up to 18 leg joints during stance when walking forward or backward over uneven surfaces and negotiation of curves, dealing with leg loss, as well as being able following motion trajectories without explicit precalculation. The different Walknet versions are compared to other approaches describing insect-inspired hexapod walking. Finally, we briefly address the ability of this decentralized reactive controller to form the basis for the simulation of higher-level cognitive faculties exceeding the capabilities of insects.

How and to what end may consciousness contribute to action? Attributing properties of consciousness to an embodied, minimally cognitive artificial neural network.

An artificial neural network called reaCog is described which is based on a decentralized, reactive and embodied architecture developed to control non-trivial hexapod walking in an unpredictable environment (Walknet) while using insect-like navigation (Navinet). In reaCog, these basic networks are extended in such a way that the complete system, reaCog, adopts the capability of inventing new behaviors and - via internal simulation - of planning ahead. This cognitive expansion enables the reactive system to be enriched with additional procedures. Here, we focus on the question to what extent properties of phenomena to be characterized on a different level of description as for example consciousness can be found in this minimally cognitive system. Adopting a monist view, we argue that the phenomenal aspect of mental phenomena can be neglected when discussing the function of such a system. Under this condition, reaCog is discussed to be equipped with properties as are bottom-up and top-down attention, intentions, volition, and some aspects of Access Consciousness. These properties have not been explicitly implemented but emerge from the cooperation between the elements of the network. The aspects of Access Consciousness found in reaCog concern the above mentioned ability to plan ahead and to invent and guide (new) actions. Furthermore, global accessibility of memory elements, another aspect characterizing Access Consciousness is realized by this network. reaCog allows for both reactive/automatic control and (access-) conscious control of behavior. We discuss examples for interactions between both the reactive domain and the conscious domain. Metacognition or Reflexive Consciousness is not a property of reaCog. Possible expansions are discussed to allow for further properties of Access Consciousness, verbal report on internal states, and for Metacognition. In summary, we argue that already simple networks allow for properties of consciousness if leaving the phenomenal aspect aside.

What's Next: Recruitment of a Grounded Predictive Body Model for Planning a Robot's Actions.

Even comparatively simple, reactive systems are able to control complex motor tasks, such as hexapod walking on unpredictable substrate. The capability of such a controller can be improved by introducing internal models of the body and of parts of the environment. Such internal models can be applied as inverse models, as forward models or to solve the problem of sensor fusion. Usually, separate models are used for these functions. Furthermore, separate models are used to solve different tasks. Here we concentrate on internal models of the body as the brain considers its own body the most important part of the world. The model proposed is formed by a recurrent neural network with the property of pattern completion. The model shows a hierarchical structure but nonetheless comprises a holistic system. One and the same model can be used as a forward model, as an inverse model, for sensor fusion, and, with a simple expansion, as a model to internally simulate (new) behaviors to be used for prediction. The model embraces the geometrical constraints of a complex body with many redundant degrees of freedom, and allows finding geometrically possible solutions. To control behavior such as walking, climbing, or reaching, this body model is complemented by a number of simple reactive procedures together forming a procedural memory. In this article, we illustrate the functioning of this network. To this end we present examples for solutions of the forward function and the inverse function, and explain how the complete network might be used for predictive purposes. The model is assumed to be "innate," so learning the parameters of the model is not (yet) considered.

Hexapod Walking: an expansion to Walknet dealing with leg amputations and force oscillations.

The control of the legs of a walking hexapod is a complex problem as the legs have three joints each, resulting in a total of 18 degrees of freedom. We addressed this problem using a decentralized architecture termed Walknet, which consists of peripheral pattern generators being coordinated through influences acting mainly between neighbouring legs. Both, the coordinating influences and the local control modules (each acting only on one leg), are biologically inspired. This investigation shows that it is possible to adapt this approach to account for additional biological data by (1) changing the structure of the selector net in a biological plausible way (including force as an analog variable), (2) introducing a biologically motivated coordination influence for coactivation between legs and (3) adding a hypothetical influence between hind and front legs. This network of controllers has been tested using a dynamic simulation. It is able to describe (a) the behaviour of animals walking with one or two legs being amputated and (b) force oscillations that occur in a specific experimental situation, the standing legs of a walking animal.