Recurrent neural networks that learn multi-step visual routines with reinforcement learning.

Many cognitive problems can be decomposed into series of subproblems that are solved sequentially by the brain. When subproblems are solved, relevant intermediate results need to be stored by neurons and propagated to the next subproblem, until the overarching goal has been completed. We will here consider visual tasks, which can be decomposed into sequences of elemental visual operations. Experimental evidence suggests that intermediate results of the elemental operations are stored in working memory as an enhancement of neural activity in the visual cortex. The focus of enhanced activity is then available for subsequent operations to act upon. The main question at stake is how the elemental operations and their sequencing can emerge in neural networks that are trained with only rewards, in a reinforcement learning setting. We here propose a new recurrent neural network architecture that can learn composite visual tasks that require the application of successive elemental operations. Specifically, we selected three tasks for which electrophysiological recordings of monkeys' visual cortex are available. To train the networks, we used RELEARNN, a biologically plausible four-factor Hebbian learning rule, which is local both in time and space. We report that networks learn elemental operations, such as contour grouping and visual search, and execute sequences of operations, solely based on the characteristics of the visual stimuli and the reward structure of a task. After training was completed, the activity of the units of the neural network elicited by behaviorally relevant image items was stronger than that elicited by irrelevant ones, just as has been observed in the visual cortex of monkeys solving the same tasks. Relevant information that needed to be exchanged between subroutines was maintained as a focus of enhanced activity and passed on to the subsequent subroutines. Our results demonstrate how a biologically plausible learning rule can train a recurrent neural network on multistep visual tasks.

A neuronal basis of iconic memory in macaque primary visual cortex.

After a briefly presented visual stimulus disappears, observers retain a detailed representation of this stimulus for a short period of time. This sensory storage is called iconic memory. We measured iconic memory in the perception of monkeys and its neuronal correlates in the primary visual cortex (area V1). We determined how many milliseconds extra viewing time iconic memory is worth and how it decays by varying the duration of a brief stimulus and the timing of a mask. The V1 activity that persists after the disappearance of a stimulus predicted accuracy, with a time course resembling the worth and decay of iconic memory. Finally, we examined how iconic memory interacts with attention. A cue presented after the stimulus disappears boosts attentional influences pertaining to a relevant part of the stimulus but only if it appears before iconic memory decayed. Our results relate iconic memory to neuronal activity in early visual cortex.

Impairment in predictive processes during auditory mismatch negativity in ScZ: Evidence from event-related fields.

Patients with schizophrenia (ScZ) show pronounced dysfunctions in auditory perception but the underlying mechanisms as well as the localization of the deficit remain unclear. To examine these questions, the current study examined whether alterations in the neuromagnetic mismatch negativity (MMNm) in ScZ-patients could involve an impairment in sensory predictions in local sensory and higher auditory areas. Using a whole-head MEG-approach, we investigated the MMNm as well as P300m and N100m amplitudes during a hierarchical auditory novelty paradigm in 16 medicated ScZ-patients and 16 controls. In addition, responses to omitted sounds were investigated, allowing for a critical test of the predictive coding hypothesis. Source-localization was performed to identify the generators of the MMNm, omission responses as well as the P300m. Clinical symptoms were examined with the positive and negative syndrome scale. Event-related fields (ERFs) to standard sounds were intact in ScZ-patients. However, the ScZ-group showed a reduction in the amplitude of the MMNm during both local (within trials) and global (across trials) conditions as well as an absent P300m at the global level. Importantly, responses to sound omissions were reduced in ScZ-patients which overlapped both in latency and generators with the MMNm sources. Thus, our data suggest that auditory dysfunctions in ScZ involve impaired predictive processes that involve deficits in both automatic and conscious detection of auditory regularities. Hum Brain Mapp 38:5082-5093, 2017. © 2017 Wiley Periodicals, Inc.

3D printing and modelling of customized implants and surgical guides for non-human primates.

BACKGROUND: Primate neurobiologists use chronically implanted devices such as pedestals for head stabilization and chambers to gain access to the brain and study its activity. Such implants are skull-mounted, and made from a hard, durable material, such as titanium. NEW METHOD: Here, we present a low-cost method of creating customized 3D-printed cranial implants that are tailored to the anatomy of individual animals. We performed pre-surgical computed tomography (CT) and magnetic resonance (MR) scans to generate three-dimensional (3D) models of the skull and brain. We then used 3D modelling software to design implantable head posts, chambers, and a pedestal anchorage base, as well as craniotomy guides to aid us during surgery. Prototypes were made from plastic or resin, while implants were 3D-printed in titanium. The implants underwent post-processing and received a coating of osteocompatible material to promote bone integration. RESULTS: Their tailored fit greatly facilitated surgical implantation, and eliminated the gap between the implant and the bone. To date, our implants remain robust and well-integrated with the skull. COMPARISON WITH EXISTING METHOD(S): Commercial-off-the-shelf solutions typically come with a uniform, flat base, preventing them from sitting flush against the curved surface of the skull. This leaves gaps for fluid and tissue ingress, increasing the risk of microbial infection and tissue inflammation, as well as implant loss. CONCLUSIONS: The use of 3D printing technology enabled us to quickly and affordably create unique, complex designs, avoiding the constraints levied by traditional production methods, thereby boosting experimental success and improving the wellbeing of the animals.

A predictive coding account of MMN reduction in schizophrenia.

The mismatch negativity (MMN) is thought to be an index of the automatic activation of a specialized network for active prediction and deviance detection in the auditory cortex. It is consistently reduced in schizophrenic patients and has received a lot of interest as a clinical and translational tool. The main neuronal hypothesis regarding the mechanisms leading to a reduced MMN in schizophrenic patients is a dysfunction of NMDA receptors (NMDA-R). However, this hypothesis has never been implemented in a neuronal model. In this paper, we examine the consequences of NMDA-R dysfunction in a neuronal model of MMN based on predictive coding principle. I also investigate how predictive processes may interact with synaptic adaptation in MMN generations and examine the consequences of this interaction for the use of MMN paradigms in schizophrenia research.

The Neural Representation of Sequences: From Transition Probabilities to Algebraic Patterns and Linguistic Trees.

A sequence of images, sounds, or words can be stored at several levels of detail, from specific items and their timing to abstract structure. We propose a taxonomy of five distinct cerebral mechanisms for sequence coding: transitions and timing knowledge, chunking, ordinal knowledge, algebraic patterns, and nested tree structures. In each case, we review the available experimental paradigms and list the behavioral and neural signatures of the systems involved. Tree structures require a specific recursive neural code, as yet unidentified by electrophysiology, possibly unique to humans, and which may explain the singularity of human language and cognition.

A neuronal model of predictive coding accounting for the mismatch negativity.

The mismatch negativity (MMN) is thought to index the activation of specialized neural networks for active prediction and deviance detection. However, a detailed neuronal model of the neurobiological mechanisms underlying the MMN is still lacking, and its computational foundations remain debated. We propose here a detailed neuronal model of auditory cortex, based on predictive coding, that accounts for the critical features of MMN. The model is entirely composed of spiking excitatory and inhibitory neurons interconnected in a layered cortical architecture with distinct input, predictive, and prediction error units. A spike-timing dependent learning rule, relying upon NMDA receptor synaptic transmission, allows the network to adjust its internal predictions and use a memory of the recent past inputs to anticipate on future stimuli based on transition statistics. We demonstrate that this simple architecture can account for the major empirical properties of the MMN. These include a frequency-dependent response to rare deviants, a response to unexpected repeats in alternating sequences (ABABAA…), a lack of consideration of the global sequence context, a response to sound omission, and a sensitivity of the MMN to NMDA receptor antagonists. Novel predictions are presented, and a new magnetoencephalography experiment in healthy human subjects is presented that validates our key hypothesis: the MMN results from active cortical prediction rather than passive synaptic habituation.

Evidence for a hierarchy of predictions and prediction errors in human cortex.

According to hierarchical predictive coding models, the cortex constantly generates predictions of incoming stimuli at multiple levels of processing. Responses to auditory mismatches and omissions are interpreted as reflecting the prediction error when these predictions are violated. An alternative interpretation, however, is that neurons passively adapt to repeated stimuli. We separated these alternative interpretations by designing a hierarchical auditory novelty paradigm and recording human EEG and magnetoencephalographic (MEG) responses to mismatching or omitted stimuli. In the crucial condition, participants listened to frequent series of four identical tones followed by a fifth different tone, which generates a mismatch response. Because this response itself is frequent and expected, the hierarchical predictive coding hypothesis suggests that it should be cancelled out by a higher-order prediction. Three consequences ensue. First, the mismatch response should be larger when it is unexpected than when it is expected. Second, a perfectly monotonic sequence of five identical tones should now elicit a higher-order novelty response. Third, omitting the fifth tone should reveal the brain's hierarchical predictions. The rationale here is that, when a deviant tone is expected, its omission represents a violation of two expectations: a local prediction of a tone plus a hierarchically higher expectation of its deviancy. Thus, such an omission should induce a greater prediction error than when a standard tone is expected. Simultaneous EEE- magnetoencephalographic recordings verify those predictions and thus strongly support the predictive coding hypothesis. Higher-order predictions appear to be generated in multiple areas of frontal and associative cortices.

Motor control by sensory cortex.

Classical studies of mammalian movement control define a prominent role for the primary motor cortex. Investigating the mouse whisker system, we found an additional and equally direct pathway for cortical motor control driven by the primary somatosensory cortex. Whereas activity in primary motor cortex directly evokes exploratory whisker protraction, primary somatosensory cortex directly drives whisker retraction, providing a rapid negative feedback signal for sensorimotor integration. Motor control by sensory cortex suggests the need to reevaluate the functional organization of cortical maps.