Is a 4-bit synaptic weight resolution enough? - constraints on enabling spike-timing dependent plasticity in neuromorphic hardware.

Large-scale neuromorphic hardware systems typically bear the trade-off between detail level and required chip resources. Especially when implementing spike-timing dependent plasticity, reduction in resources leads to limitations as compared to floating point precision. By design, a natural modification that saves resources would be reducing synaptic weight resolution. In this study, we give an estimate for the impact of synaptic weight discretization on different levels, ranging from random walks of individual weights to computer simulations of spiking neural networks. The FACETS wafer-scale hardware system offers a 4-bit resolution of synaptic weights, which is shown to be sufficient within the scope of our network benchmark. Our findings indicate that increasing the resolution may not even be useful in light of further restrictions of customized mixed-signal synapses. In addition, variations due to production imperfections are investigated and shown to be uncritical in the context of the presented study. Our results represent a general framework for setting up and configuring hardware-constrained synapses. We suggest how weight discretization could be considered for other backends dedicated to large-scale simulations. Thus, our proposition of a good hardware verification practice may rise synergy effects between hardware developers and neuroscientists.

A compositionality machine realized by a hierarchic architecture of synfire chains.

The composition of complex behavior is thought to rely on the concurrent and sequential activation of simpler action components, or primitives. Systems of synfire chains have previously been proposed to account for either the simultaneous or the sequential aspects of compositionality; however, the compatibility of the two aspects has so far not been addressed. Moreover, the simultaneous activation of primitives has up until now only been investigated in the context of reactive computations, i.e., the perception of stimuli. In this study we demonstrate how a hierarchical organization of synfire chains is capable of generating both aspects of compositionality for proactive computations such as the generation of complex and ongoing action. To this end, we develop a network model consisting of two layers of synfire chains. Using simple drawing strokes as a visualization of abstract primitives, we map the feed-forward activity of the upper level synfire chains to motion in two-dimensional space. Our model is capable of producing drawing strokes that are combinations of primitive strokes by binding together the corresponding chains. Moreover, when the lower layer of the network is constructed in a closed-loop fashion, drawing strokes are generated sequentially. The generated pattern can be random or deterministic, depending on the connection pattern between the lower level chains. We propose quantitative measures for simultaneity and sequentiality, revealing a wide parameter range in which both aspects are fulfilled. Finally, we investigate the spiking activity of our model to propose candidate signatures of synfire chain computation in measurements of neural activity during action execution.

Cortext: a columnar model of bottom-up and top-down processing in the neocortex.

Experimental data suggests that a first hypothesis about the content of a complex visual scene is available as early as 150 ms after stimulus presentation. Other evidence suggests that recognition in the visual cortex of mammals is a bidirectional, often top-down driven process. Here, we present a spiking neural network model that demonstrates how the cortex can use both strategies: Faced with a new stimulus, the cortex first tries to catch the gist of the scene. The gist is then fed back as global hypothesis to influence and redirect further bottom-up processing. We propose that these two modes of processing are carried out in different layers of the cortex. A cortical column may, thus, be primarily defined by the specific connectivity that links neurons in different layers into a functional circuit. Given an input, our model generates an initial hypothesis after only a few milliseconds. The first wave of action potentials traveling up the hierarchy activates representations of features and feature combinations. In most cases, the correct feature representation is activated strongest and precedes all other candidates with millisecond precision. Thus, our model codes the reliability of a response in the relative latency of spikes. In the subsequent refinement stage where high-level activity modulates lower stages, this activation dominance is propagated back, influencing its own afferent activity to establish a unique decision. Thus, top-down influence de-activates representations that have contributed to the initial hypothesis about the current stimulus, comparable to predictive coding. Features that do not match the top-down prediction trigger an error signal that can be the basis for learning new representations.

Detecting synfire chain activity using massively parallel spike train recording.

The synfire chain model has been proposed as the substrate that underlies computational processes in the brain and has received extensive theoretical study. In this model cortical tissue is composed of a superposition of feedforward subnetworks (chains) each capable of transmitting packets of synchronized spikes with high reliability. Computations are then carried out by interactions of these chains. Experimental evidence for synfire chains has so far been limited to inference from detection of a few repeating spatiotemporal neuronal firing patterns in multiple single-unit recordings. Demonstration that such patterns actually come from synfire activity would require finding a meta organization among many detected patterns, as yet an untried approach. In contrast we present here a new method that directly visualizes the repetitive occurrence of synfire activity even in very large data sets of multiple single-unit recordings. We achieve reliability and sensitivity by appropriately averaging over neuron space (identities) and time. We test the method with data from a large-scale balanced recurrent network simulation containing 50 randomly activated synfire chains. The sensitivity is high enough to detect synfire chain activity in simultaneous single-unit recordings of 100 to 200 neurons from such data, enabling application to experimental data in the near future.

The high-conductance state of cortical networks.

We studied the dynamics of large networks of spiking neurons with conductance-based (nonlinear) synapses and compared them to networks with current-based (linear) synapses. For systems with sparse and inhibition-dominated recurrent connectivity, weak external inputs induced asynchronous irregular firing at low rates. Membrane potentials fluctuated a few millivolts below threshold, and membrane conductances were increased by a factor 2 to 5 with respect to the resting state. This combination of parameters characterizes the ongoing spiking activity typically recorded in the cortex in vivo. Many aspects of the asynchronous irregular state in conductance-based networks could be sufficiently well characterized with a simple numerical mean field approach. In particular, it correctly predicted an intriguing property of conductance-based networks that does not appear to be shared by current-based models: they exhibit states of low-rate asynchronous irregular activity that persist for some period of time even in the absence of external inputs and without cortical pacemakers. Simulations of larger networks (up to 350,000 neurons) demonstrated that the survival time of self-sustained activity increases exponentially with network size.