Imitation and mirror systems in robots through Deep Modality Blending Networks.

Learning to interact with the environment not only empowers the agent with manipulation capability but also generates information to facilitate building of action understanding and imitation capabilities. This seems to be a strategy adopted by biological systems, in particular primates, as evidenced by the existence of mirror neurons that seem to be involved in multi-modal action understanding. How to benefit from the interaction experience of the robots to enable understanding actions and goals of other agents is still a challenging question. In this study, we propose a novel method, deep modality blending networks (DMBN), that creates a common latent space from multi-modal experience of a robot by blending multi-modal signals with a stochastic weighting mechanism. We show for the first time that deep learning, when combined with a novel modality blending scheme, can facilitate action recognition and produce structures to sustain anatomical and effect-based imitation capabilities. Our proposed system, which is based on conditional neural processes, can be conditioned on any desired sensory/motor value at any time step, and can generate a complete multi-modal trajectory consistent with the desired conditioning in one-shot by querying the network for all the sampled time points in parallel avoiding the accumulation of prediction errors. Based on simulation experiments with an arm-gripper robot and an RGB camera, we showed that DMBN could make accurate predictions about any missing modality (camera or joint angles) given the available ones outperforming recent multimodal variational autoencoder models in terms of long-horizon high-dimensional trajectory predictions. We further showed that given desired images from different perspectives, i.e. images generated by the observation of other robots placed on different sides of the table, our system could generate image and joint angle sequences that correspond to either anatomical or effect-based imitation behavior. To achieve this mirror-like behavior, our system does not perform a pixel-based template matching but rather benefits from and relies on the common latent space constructed by using both joint and image modalities, as shown by additional experiments. Moreover, we showed that mirror learning (in our system) does not only depend on visual experience and cannot be achieved without proprioceptive experience. Our experiments showed that out of ten training scenarios with different initial configurations, the proposed DMBN model could achieve mirror learning in all of the cases where the model that only uses visual information failed in half of them. Overall, the proposed DMBN architecture not only serves as a computational model for sustaining mirror neuron-like capabilities, but also stands as a powerful machine learning architecture for high-dimensional multi-modal temporal data with robust retrieval capabilities operating with partial information in one or multiple modalities.

Human-In-The-Loop Control and Task Learning for Pneumatically Actuated Muscle Based Robots.

Pneumatically actuated muscles (PAMs) provide a low cost, lightweight, and high power-to-weight ratio solution for many robotic applications. In addition, the antagonist pair configuration for robotic arms make it open to biologically inspired control approaches. In spite of these advantages, they have not been widely adopted in human-in-the-loop control and learning applications. In this study, we propose a biologically inspired multimodal human-in-the-loop control system for driving a one degree-of-freedom robot, and realize the task of hammering a nail into a wood block under human control. We analyze the human sensorimotor learning in this system through a set of experiments, and show that effective autonomous hammering skill can be readily obtained through the developed human-robot interface. The results indicate that a human-in-the-loop learning setup with anthropomorphically valid multi-modal human-robot interface leads to fast learning, thus can be used to effectively derive autonomous robot skills for ballistic motor tasks that require modulation of impedance.

Shoulder Glenohumeral Elevation Estimation based on Upper Arm Orientation.

In this paper, the shoulder glenohumeral displacement during the movement of the upper arm is studied. Four modeling approaches were examined and compared to estimate the humeral head elevation (vertical displacement) and translation (horizontal displacement). A biomechanics-inspired method was used firstly to model the glenohumeral displacement in which a least squares method was implemented for parameter identification. Then, three Gaussian process regression models were used in which the following variable sets were employed: i) shoulder adduction/abduction angle, ii) combination of shoulder adduction/abduction and flexion/extension angles, iii) overall upper arm orientation in the form of quaternions. In order to test the respective performances of these four models, we collected motion capture data and compared the models' representative capabilities. As a result, Gaussian process regression that considered the overall upper arm orientation outperformed the other modeling approaches; however, it should be noted that the other methods also provided accuracy levels that may be sufficient depending on task requirements.

Human motor adaptation in whole body motion.

The main role of the sensorimotor system of an organism is to increase the survival of the species. Therefore, to understand the adaptation and optimality mechanisms of motor control, it is necessary to study the sensorimotor system in terms of ecological fitness. We designed an experimental paradigm that exposed sensorimotor system to risk of injury. We studied human subjects performing uncon- strained squat-to-stand movements that were systematically subjected to non-trivial perturbation. We found that subjects adapted by actively compensating the perturbations, converging to movements that were different from their normal unperturbed squat-to-stand movements. Furthermore, the adapted movements had clear intrinsic inter-subject differences which could be explained by different adapta- tion strategies employed by the subjects. These results suggest that classical optimality measures of physical energy and task satisfaction should be seen as part of a hierarchical organization of optimality with safety being at the highest level. Therefore, in addition to physical energy and task fulfillment, the risk of injury and other possible costs such as neural computational overhead have to be considered when analyzing human movement.

Minimal Sign Representation of Boolean Functions: Algorithms and Exact Results for Low Dimensions.

Boolean functions (BFs) are central in many fields of engineering and mathematics, such as cryptography, circuit design, and combinatorics. Moreover, they provide a simple framework for studying neural computation mechanisms of the brain. Many representation schemes for BFs exist to satisfy the needs of the domain they are used in. In neural computation, it is of interest to know how many input lines a neuron would need to represent a given BF. A common BF representation to study this is the so-called polynomial sign representation where [Formula: see text] and 1 are associated with true and false, respectively. The polynomial is treated as a real-valued function and evaluated at its parameters, and the sign of the polynomial is then taken as the function value. The number of input lines for the modeled neuron is exactly the number of terms in the polynomial. This letter investigates the minimum number of terms, that is, the minimum threshold density, that is sufficient to represent a given BF and more generally aims to find the maximum over this quantity for all BFs in a given dimension. With this work, for the first time exact results for four- and five-variable BFs are obtained, and strong bounds for six-variable BFs are derived. In addition, some connections between the sign representation framework and bent functions are derived, which are generally studied for their desirable cryptographic properties.

Action and language mechanisms in the brain: data, models and neuroinformatics.

We assess the challenges of studying action and language mechanisms in the brain, both singly and in relation to each other to provide a novel perspective on neuroinformatics, integrating the development of databases for encoding ­ separately or together ­ neurocomputational models and empirical data that serve systems and cognitive neuroscience.

Mirror neurons: functions, mechanisms and models.

Mirror neurons for manipulation fire both when the animal manipulates an object in a specific way and when it sees another animal (or the experimenter) perform an action that is more or less similar. Such neurons were originally found in macaque monkeys, in the ventral premotor cortex, area F5 and later also in the inferior parietal lobule. Recent neuroimaging data indicate that the adult human brain is endowed with a "mirror neuron system," putatively containing mirror neurons and other neurons, for matching the observation and execution of actions. Mirror neurons may serve action recognition in monkeys as well as humans, whereas their putative role in imitation and language may be realized in human but not in monkey. This article shows the important role of computational models in providing sufficient and causal explanations for the observed phenomena involving mirror systems and the learning processes which form them, and underlines the need for additional circuitry to lift up the monkey mirror neuron circuit to sustain the posited cognitive functions attributed to the human mirror neuron system.

Sign-representation of Boolean functions using a small number of monomials.

This paper presents a deterministic algorithm that can construct a higher-order neuron representation for an arbitrary n-variable Boolean function with a fan-in less than 0.75 x 2(n), and provides related theoretical results. When the logic constants True and False are identified by +1 and -1, an n-variable Boolean function is identified by a unique dichotomy of the n-dimensional hypercube. With this equivalence, all n-variable Boolean functions can be uniquely represented by linear combinations of monomials, the products of input variables. A polynomial function whose sign matches the truth table of a given Boolean function is said to sign-represent that Boolean function. The artificial neural units that implement this sign-representation scheme are often called higher-order neurons or polynomial threshold units. This paper investigates the freedom provided by the sign-representation framework in terms of the fan-in of these artificial neural units. In particular, we look for sign-representations with a small number of monomials. Although there are methods developed for finding a reduced set of monomials to represent Boolean functions, there are no deterministic algorithms for computing non-trivial solutions with guarantees on the number of monomials in the found sign-representations. This work fills this gap by providing deterministic algorithms which are guaranteed to find solutions with fewer than 0.75 x 2(n) monomials for n-variable Boolean functions. Although the algorithms presented here are computationally costly, it is expected that several research directions can be spawned from the current study, such as reducing the 0.75 x 2(n) bound and devising efficient algorithms for finding sign-representations with a small number of monomials.

From self-observation to imitation: visuomotor association on a robotic hand.

Being at the crux of human cognition and behaviour, imitation has become the target of investigations ranging from experimental psychology and neurophysiology to computational sciences and robotics. It is often assumed that the imitation is innate, but it has more recently been argued, both theoretically and experimentally, that basic forms of imitation could emerge as a result of self-observation. Here, we tested this proposal on a realistic experimental platform, comprising an associative network linking a 16 degrees of freedom robotic hand and a simple visual system. We report that this minimal visuomotor association is sufficient to bootstrap basic imitation. Our results indicate that crucial features of human imitation, such as generalization to new actions, may emerge from a connectionist associative network. Therefore, we suggest that a behaviour as complex as imitation could be, at the neuronal level, founded on basic mechanisms of associative learning, a notion supported by a recent proposal on the developmental origin of mirror neurons. Our approach can be applied to the development of realistic cognitive architectures for humanoid robots as well as to shed new light on the cognitive processes at play in early human cognitive development.

An upper bound on the minimum number of monomials required to separate dichotomies of {-1, 1}n.

It is known that any dichotomy of {-1, 1}n can be learned (separated) with a higher-order neuron (polynomial function) with 2n inputs (monomials). In general, less than 2n monomials are sufficient to solve a given dichotomy. In spite of the efforts to develop algorithms for finding solutions with fewer monomials, there have been relatively fewer studies investigating maximum density (Pi(n)), the minimum number of monomials that would suffice to separate an arbitrary dichotomy of {-1, 1}n . This article derives a theoretical (upper) bound for this quantity, superseding previously known bounds. The main theorem here states that for any binary classification problem in {-1, 1}n (n > 1), one can always find a polynomial function solution with 2n -2n/4 or fewer monomials. In particular, any dichotomy of {-1, 1}n can be learned by a higher-order neuron with a fan-in of 2n -2n/4 or less. With this result, for the first time, a deterministic ratio bound independent of n is established as Pi(n)/2n < or = 0 75. The main theorem is constructive, so it provides a deterministic algorithm for achieving the theoretical result. The study presented provides the basic mathematical tools and forms the basis for further analyses that may have implications for neural computation mechanisms employed in the cerebral cortex.

Mirror neurons and imitation: a computationally guided review.

Neurophysiology reveals the properties of individual mirror neurons in the macaque while brain imaging reveals the presence of 'mirror systems' (not individual neurons) in the human. Current conceptual models attribute high level functions such as action understanding, imitation, and language to mirror neurons. However, only the first of these three functions is well-developed in monkeys. We thus distinguish current opinions (conceptual models) on mirror neuron function from more detailed computational models. We assess the strengths and weaknesses of current computational models in addressing the data and speculations on mirror neurons (macaque) and mirror systems (human). In particular, our mirror neuron system (MNS), mental state inference (MSI) and modular selection and identification for control (MOSAIC) models are analyzed in more detail. Conceptual models often overlook the computational requirements for posited functions, while too many computational models adopt the erroneous hypothesis that mirror neurons are interchangeable with imitation ability. Our meta-analysis underlines the gap between conceptual and computational models and points out the research effort required from both sides to reduce this gap.

Mental state inference using visual control parameters.

Although we can often infer the mental states of others by observing their actions, there are currently no computational models of this remarkable ability. Here we develop a computational model of mental state inference that builds upon a generic visuomanual feedback controller, and implements mental simulation and mental state inference functions using circuitry that subserves sensorimotor control. Our goal is (1) to show that control mechanisms developed for manual manipulation are readily endowed with visual and predictive processing capabilities and thus allows a natural extension to the understanding of movements performed by others; and (2) to give an explanation on how cortical regions, in particular the parietal and premotor cortices, may be involved in such dual mechanism. To analyze the model, we simulate tasks in which an observer watches an actor performing either a reaching or a grasping movement. The observer's goal is to estimate the 'mental state' of the actor: the goal of the reaching movement or the intention of the agent performing the grasping movement. We show that the motor modules of the observer can be used in a 'simulation mode' to infer the mental state of the actor. The simulations with different grasping and non-straight line reaching strategies show that the mental state inference model is applicable to complex movements. Moreover, we simulate deceptive reaching, where an actor imposes false beliefs about his own mental state on an observer. The simulations show that computational elements developed for sensorimotor control are effective in inferring the mental states of others. The parallels between the model and cortical organization of movement suggest that primates might have developed a similar resource utilization strategy for action understanding, and thus lead to testable predictions about the brain mechanisms of mental state inference.

Infant grasp learning: a computational model.

This paper presents ILGM (the Infant Learning to Grasp Model), the first computational model of infant grasp learning that is constrained by the infant motor development literature. By grasp learning we mean learning how to make motor plans in response to sensory stimuli such that open-loop execution of the plan leads to a successful grasp. The open-loop assumption is justified by the behavioral evidence that early grasping is based on open-loop control rather than on-line visual feedback. Key elements of the infancy period, namely elementary motor schemas, the exploratory nature of infant motor interaction, and inherent motor variability are captured in the model. In particular we show, through computational modeling, how an existing behavior (reaching) yields a more complex behavior (grasping) through interactive goal-directed trial and error learning. Our study focuses on how the infant learns to generate grasps that match the affordances presented by objects in the environment. ILGM was designed to learn execution parameters for controlling the hand movement as well as for modulating the reach to provide a successful grasp matching the target object affordance. Moreover, ILGM produces testable predictions regarding infant motor learning processes and poses new questions to experimentalists.

Schema design and implementation of the grasp-related mirror neuron system.

Mirror neurons within a monkey's premotor area F5 fire not only when the monkey performs a certain class of actions but also when the monkey observes another monkey (or the experimenter) perform a similar action. It has thus been argued that these neurons are crucial for understanding of actions by others. We offer the hand-state hypothesis as a new explanation of the evolution of this capability: the basic functionality of the F5 mirror system is to elaborate the appropriate feedback - what we call the hand state - for opposition-space based control of manual grasping of an object. Given this functionality, the social role of the F5 mirror system in understanding the actions of others may be seen as an exaptation gained by generalizing from one's own hand to an other's hand. In other words, mirror neurons first evolved to augment the "canonical" F5 neurons (active during self-movement based on observation of an object) by providing visual feedback on "hand state," relating the shape of the hand to the shape of the object. We then introduce the MNS1 (mirror neuron system 1) model of F5 and related brain regions. The existing Fagg-Arbib-Rizzolatti-Sakata model represents circuitry for visually guided grasping of objects, linking the anterior intraparietal area (AIP) with F5 canonical neurons. The MNS1 model extends the AIP visual pathway by also modeling pathways, directed toward F5 mirror neurons, which match arm-hand trajectories to the affordances and location of a potential target object. We present the basic schemas for the MNS1 model, then aggregate them into three "grand schemas" - visual analysis of hand state, reach and grasp, and the core mirror circuit - for each of which we present a useful implementation (a non-neural visual processing system, a multijoint 3-D kinematics simulator, and a learning neural network, respectively). With this implementation we show how the mirror system may learn to recognize actions already in the repertoire of the F5 canonical neurons. We show that the connectivity pattern of mirror neuron circuitry can be established through training, and that the resultant network can exhibit a range of novel, physiologically interesting behaviors during the process of action recognition. We train the system on the basis of final grasp but then observe the whole time course of mirror neuron activity, yielding predictions for neurophysiological experiments under conditions of spatial perturbation, altered kinematics, and ambiguous grasp execution which highlight the importance of the timing of mirror neuron activity.