The DIAMOND Model: Deep Recurrent Neural Networks for Self-Organizing Robot Control.

The proposed architecture applies the principle of predictive coding and deep learning in a brain-inspired approach to robotic sensorimotor control. It is composed of many layers each of which is a recurrent network. The component networks can be spontaneously active due to the homeokinetic learning rule, a principle that has been studied previously for the purpose of self-organized generation of behavior. We present robotic simulations that illustrate the function of the network and show evidence that deeper networks enable more complex exploratory behavior.

Local Autoencoding for Parameter Estimation in a Hidden Potts-Markov Random Field.

A local-autoencoding (LAE) method is proposed for the parameter estimation in a Hidden Potts-Markov random field model. Due to sampling cost, Markov chain Monte Carlo methods are rarely used in real-time applications. Like other heuristic methods, LAE is based on a conditional independence assumption. It adapts, however, the parameters in a block-by-block style with a simple Hebbian learning rule. Experiments with given label fields show that the LAE is able to converge in far less time than required for a scan. It is also possible to derive an estimate for LAE based on a Cramer­Rao bound that is similar to the classical maximum pseudolikelihood method. As a general algorithm, LAE can be used to estimate the parameters in anisotropic label fields. Furthermore, LAE is not limited to the classical Potts model and can be applied to other types of Potts models by simple label field transformations and straightforward learning rule extensions. Experimental results on image segmentations demonstrate the efficiency and generality of the LAE algorithm.

Cell-Division Behavior in a Heterogeneous Swarm Environment.

We present a system of virtual particles that interact using simple kinetic rules. It is known that heterogeneous mixtures of particles can produce particularly interesting behaviors. Here we present a two-species three-dimensional swarm in which a behavior emerges that resembles cell division. We show that the dividing behavior exists across a narrow but finite band of parameters and for a wide range of population sizes. When executed in a two-dimensional environment the swarm's characteristics and dynamism manifest differently. In further experiments we show that repeated divisions can occur if the system is extended by a biased equilibrium process to control the split of populations. We propose that this repeated division behavior provides a simple model for cell-division mechanisms and is of interest for the formation of morphological structure and to swarm robotics.

Natural Vocalizations in the Mammalian Inferior Colliculus are Broadly Encoded by a Small Number of Independent Multi-Units.

How complex natural sounds are represented by the main converging center of the auditory midbrain, the central inferior colliculus, is an open question. We applied neural discrimination to determine the variation of detailed encoding of individual vocalizations across the best frequency gradient of the central inferior colliculus. The analysis was based on collective responses from several neurons. These multi-unit spike trains were recorded from guinea pigs exposed to a spectrotemporally rich set of eleven species-specific vocalizations. Spike trains of disparate units from the same recording were combined in order to investigate whether groups of multi-unit clusters represent the whole set of vocalizations more reliably than only one unit, and whether temporal response correlations between them facilitate an unambiguous neural representation of the vocalizations. We found a spatial distribution of the capability to accurately encode groups of vocalizations across the best frequency gradient. Different vocalizations are optimally discriminated at different locations of the best frequency gradient. Furthermore, groups of a few multi-unit clusters yield improved discrimination over only one multi-unit cluster between all tested vocalizations. However, temporal response correlations between units do not yield better discrimination. Our study is based on a large set of units of simultaneously recorded responses from several guinea pigs and electrode insertion positions. Our findings suggest a broadly distributed code for behaviorally relevant vocalizations in the mammalian inferior colliculus. Responses from a few non-interacting units are sufficient to faithfully represent the whole set of studied vocalizations with diverse spectrotemporal properties.

Critical dynamics in associative memory networks.

Critical behavior in neural networks is characterized by scale-free avalanche size distributions and can be explained by self-regulatory mechanisms. Theoretical and experimental evidence indicates that information storage capacity reaches its maximum in the critical regime. We study the effect of structural connectivity formed by Hebbian learning on the criticality of network dynamics. The network only endowed with Hebbian learning does not allow for simultaneous information storage and criticality. However, the critical regime can be stabilized by short-term synaptic dynamics in the form of synaptic depression and facilitation or, alternatively, by homeostatic adaptation of the synaptic weights. We show that a heterogeneous distribution of maximal synaptic strengths does not preclude criticality if the Hebbian learning is alternated with periods of critical dynamics recovery. We discuss the relevance of these findings for the flexibility of memory in aging and with respect to the recent theory of synaptic plasticity.

Inhibition in the dynamics of selective attention: an integrative model for negative priming.

We introduce a computational model of the negative priming (NP) effect that includes perception, memory, attention, decision making, and action. The model is designed to provide a coherent picture across competing theories of NP. The model is formulated in terms of abstract dynamics for the activations of features, their binding into object entities, their semantic categorization as well as related memories and appropriate reactions. The dynamic variables interact in a connectionist network which is shown to be adaptable to a variety of experimental paradigms. We find that selective attention can be modeled by means of inhibitory processes and by a threshold dynamics. From the necessity of quantifying the experimental paradigms, we conclude that the specificity of the experimental paradigm must be taken into account when predicting the nature of the NP effect.

Cross-modal distortion of time perception: demerging the effects of observed and performed motion.

Temporal information is often contained in multi-sensory stimuli, but it is currently unknown how the brain combines e.g. visual and auditory cues into a coherent percept of time. The existing studies of cross-modal time perception mainly support the "modality appropriateness hypothesis", i.e. the domination of auditory temporal cues over visual ones because of the higher precision of audition for time perception. However, these studies suffer from methodical problems and conflicting results. We introduce a novel experimental paradigm to examine cross-modal time perception by combining an auditory time perception task with a visually guided motor task, requiring participants to follow an elliptic movement on a screen with a robotic manipulandum. We find that subjective duration is distorted according to the speed of visually observed movement: The faster the visual motion, the longer the perceived duration. In contrast, the actual execution of the arm movement does not contribute to this effect, but impairs discrimination performance by dual-task interference. We also show that additional training of the motor task attenuates the interference, but does not affect the distortion of subjective duration. The study demonstrates direct influence of visual motion on auditory temporal representations, which is independent of attentional modulation. At the same time, it provides causal support for the notion that time perception and continuous motor timing rely on separate mechanisms, a proposal that was formerly supported by correlational evidence only. The results constitute a counterexample to the modality appropriateness hypothesis and are best explained by Bayesian integration of modality-specific temporal information into a centralized "temporal hub".

Identity negative priming: a phenomenon of perception, recognition or selection?

The present study addresses the problem whether negative priming (NP) is due to information processing in perception, recognition or selection. We argue that most NP studies confound priming and perceptual similarity of prime-probe episodes and implement a color-switch paradigm in order to resolve the issue. In a series of three identity negative priming experiments with verbal naming response, we determined when NP and positive priming (PP) occur during a trial. The first experiment assessed the impact of target color on priming effects. It consisted of two blocks, each with a different fixed target color. With respect to target color no differential priming effects were found. In Experiment 2 the target color was indicated by a cue for each trial. Here we resolved the confounding of perceptual similarity and priming condition. In trials with coinciding colors for prime and probe, we found priming effects similar to Experiment 1. However, trials with a target color switch showed such effects only in trials with role-reversal (distractor-to-target or target-to-distractor), whereas the positive priming (PP) effect in the target-repetition trials disappeared. Finally, Experiment 3 split trial processing into two phases by presenting the trial-wise color cue only after the stimulus objects had been recognized. We found recognition in every priming condition to be faster than in control trials. We were hence led to the conclusion that PP is strongly affected by perception, in contrast to NP which emerges during selection, i.e., the two effects cannot be explained by a single mechanism.

The neural representation of time: an information-theoretic perspective.

A prominent finding in psychophysical experiments on time perception is Weber's law, the linear scaling of timing errors with duration. The ability to reproduce this scaling has been taken as a criterion for the validity of neurocomputational models of time perception. However, the origin of Weber's law remains unknown, and currently only a few models generically reproduce it. Here, we use an information-theoretical framework that considers the neuronal mechanisms of time perception as stochastic processes to investigate the statistical origin of Weber's law in time perception and also its frequently observed deviations. Under the assumption that the brain is able to compute optimal estimates of time, we find that Weber's law only holds exactly if the estimate is based on temporal changes in the variance of the process. In contrast, the timing errors scale sublinearly with time if the systematic changes in the mean of a process are used for estimation, as is the case in the majority of time perception models, while estimates based on temporal correlations result in a superlinear scaling. This hierarchy of temporal information is preserved if several sources of temporal information are available. Furthermore, we consider the case of multiple stochastic processes and study the examples of a covariance-based model and a model based on synfire chains. This approach reveals that existing neurocomputational models of time perception can be classified as mean-, variance- and correlation-based processes and allows predictions about the scaling of the resulting timing errors.

Variants of guided self-organization for robot control.

Autonomous robots can generate exploratory behavior by self-organization of the sensorimotor loop. We show that the behavioral manifold that is covered in this way can be modified in a goal-dependent way without reducing the self-induced activity of the robot. We present three strategies for guided self-organization, namely by using external rewards, a problem-specific error function, or assumptions about the symmetries of the desired behavior. The strategies are analyzed for two different robots in a physically realistic simulation.

Recurrence-based estimation of time-distortion functions for ERP waveform reconstruction.

We introduce an approach to compensate for temporal distortions of repeated measurements in event-related potential research. The algorithm uses a combination of methods from nonlinear time-series analysis and is based on the construction of pairwise registration functions from cross-recurrence plots of the phase-space representations of ERP signals. The globally optimal multiple-alignment path is approximated by hierarchical cluster analysis, i.e. by iteratively combining pairs of trials according to similarity. By the inclusion of context information in form of externally acquired time markers (e.g. reaction time) into a regularization scheme, the extracted warping functions can be guided near paths that are implied by the experimental procedure. All parameters occurring in the algorithm can be optimized based on the properties of the data and there is a broad regime of parameter configurations where the algorithm produces good results. Simulations on artificial data and the analysis of ERPs from a psychophysical study demonstrate the robustness and applicability of the algorithm.

Compositionality of arm movements can be realized by propagating synchrony.

We present a biologically plausible spiking neuronal network model of free monkey scribbling that reproduces experimental findings on cortical activity and the properties of the scribbling trajectory. The model is based on the idea that synfire chains can encode movement primitives. Here, we map the propagation of activity in a chain to a linearly evolving preferred velocity, which results in parabolic segments that fulfill the two-thirds power law. Connections between chains that match the final velocity of one encoded primitive to the initial velocity of the next allow the composition of random sequences of primitives with smooth transitions. The model provides an explanation for the segmentation of the trajectory and the experimentally observed deviations of the trajectory from the parabolic shape at primitive transition sites. Furthermore, the model predicts low frequency oscillations (<10 Hz) of the motor cortex local field potential during ongoing movements and increasing firing rates of non-specific motor cortex neurons before movement onset.

Event-related brain potential correlates of identity negative priming from overlapping pictures.

Event-related potentials (ERPs) were obtained from an identity priming task, where a green target had to be selected against a superimposed red distractor. Several priming conditions were realized in a mix of control (CO), negative priming (NP), and positive priming (PP) trials. PP and NP effects in reaction times (RTs) were significant. ERP results conceptually replicate earlier findings of left-posterior P300 reduction in PP and NP trials compared to CO. This ERP effect may reflect the detection of prime-probe similarity corresponding to the concept of a retrieval cue. A novel finding concerned amplitude increase of the frontal late positive complex (LPC) in the order NP, CO, and PP. NP therefore seemed to induce brain activity related to cognitive control and/or memory processes, with reduced LPC amplitude indicating effortful processing. Overall, retrieval-based explanations of identity NP are supported.

Phase transitions towards criticality in a neural system with adaptive interactions.

We analytically describe a transition scenario to self-organized criticality (SOC) that is new for physics as well as neuroscience; it combines the criticality of first and second-order phase transitions with a SOC phase. We consider a network of pulse-coupled neurons interacting via dynamical synapses, which exhibit depression and facilitation as found in experiments. We analytically show the coexistence of a SOC phase and a subcritical phase connected by a cusp bifurcation. Switching between the two phases can be triggered by varying the intensity of noisy inputs.

Stability of localized patterns in neural fields.

We investigate two-dimensional neural fields as a model of the dynamics of macroscopic activations in a cortex-like neural system. While the one-dimensional case was treated comprehensively by Amari 30 years ago, two-dimensional neural fields are much less understood. We derive conditions for the stability for the main classes of localized solutions of the neural field equation and study their behavior beyond parameter-controlled destabilization. We show that a slight modification of the original model yields an equation whose stationary states are guaranteed to satisfy the original problem and numerically demonstrate that it admits localized noncircular solutions. Typically, however, only periodic spatial tessellations emerge on destabilization of rotationally invariant solutions.

Symmetries, non-Euclidean metrics, and patterns in a Swift-Hohenberg model of the visual cortex.

The aim of this work is to investigate the effect of the shift-twist symmetry on pattern formation processes in the visual cortex. First, we describe a generic set of Riemannian metrics of the feature space of orientation preference that obeys properties of the shift-twist, translation, and reflection symmetries. Second, these metrics are embedded in a modified Swift-Hohenberg model. As a result we get a pattern formation process that resembles the pattern formation process in the visual cortex. We focus on the final stable patterns that are regular and periodic. In a third step we analyze the influences on pattern formation using weakly nonlinear theory and mode analysis. We compare the results of the present approach with earlier models.

A neurocomputational model for optimal temporal processing.

Humans can estimate the duration of intervals of time, and psychophysical experiments show that these estimations are subject to timing errors. According to standard theories of timing, these errors increase linearly with the interval to be estimated (Weber's law), and both at longer and shorter intervals, deviations from linearity are reported. This is not easily reconciled with the accumulation of neuronal noise, which would only lead to an increase with the square root of the interval. Here, we offer a neuronal model which explains the form of the error function as a result of a constrained optimization process. The model consists of a number of synfire chains with different transmission times, which project onto a set of readout neurons. We show that an increase in the transmission time corresponds to a superlinear increase of the timing errors. Under the assumption of a fixed chain length, the experimentally observed error function emerges from optimal selection of chains for each given interval. Furthermore, we show how this optimal selection could be implemented by competitive spike-timing dependent plasticity in the connections from the chains to the readout network, and discuss implications of our model on selective temporal learning and possible neural architectures of interval timing.

Neural networks approach to clustering of activity in fMRI data.

Clusters of correlated activity in functional magnetic resonance imaging data can identify regions of interest and indicate interacting brain areas. Because the extraction of clusters is computationally complex, we apply an approximative method which is based on artificial neural networks. It allows one to find clusters of various degrees of connectivity ranging between the two extreme cases of cliques and connectivity components. We propose a criterion which allows to evaluate the relevance of such structures based on the robustness with respect to parameter variations. Exploiting the intracluster correlations, we can show that regions of substantial correlation with an external stimulus can be unambiguously separated from other activity.

Shaping of receptive fields in the visual cortex during retinal maturation.

We present a computational study of the formation of simple-cell receptive field patterns in the primary visual cortex. Based on the observation that the spatial frequency of the retinal filter increases postnatally, our results explain differences in the time course of the development of orientation selectivity in binocularly deprived and normally reared kittens. Development after eye-opening in normal animals is modelled by training with natural images, whereas in the case of binocular deprivation noise-like stimulation continues. Further, it is shown that different orientation selectivities are obtained for network models trained with natural images in contrast to random phase images of identical second order statistics. The latter finding suggests that higher-order statistics of the inputs influences development of primary visual cortex. Finally, we search for quantities that identify possible signatures of natural image statistics in order to specify the amount of constructiveness that visual experience has on the formation of receptive fields.

Pattern storage and processing in attractor networks with short-time synaptic dynamics.

Neurophysiological experiments show that the strength of synaptic connections can undergo substantial changes on a short time scale. These changes depend on the history of the presynaptic input. Using mean-field techniques, we study how short-time dynamics of synaptic connections influence the performance of attractor neural networks in terms of their memory capacity and capability to process external signals. For binary discrete-time as well as for firing rate continuous-time neural networks, the fixed points of the network dynamics are shown to be unaffected by synaptic dynamics. However, the stability of patterns changes considerably. Synaptic depression turns out to reduce the storage capacity. On the other hand, synaptic depression is shown to be advantageous for processing of pattern sequences. The analytical results on stability, size of the basins of attraction and on the switching between patterns are complemented by numerical simulations.