Comparison of electrical microstimulation artifact removal methods for high-channel-count prostheses.

BACKGROUND: Neuroprostheses are used to electrically stimulate the brain, modulate neural activity and restore sensory and motor function following injury or disease, such as blindness, paralysis, and other movement and psychiatric disorders. Recordings are often made simultaneously with stimulation, allowing the monitoring of neural signals and closed-loop control of devices. However, stimulation-evoked artifacts may obscure neural activity, particularly when stimulation and recording sites are nearby. Several methods have been developed to remove stimulation artifacts, but it remains challenging to validate and compare these methods because the 'ground-truth' of the neuronal signals may be contaminated by artifacts. NEW METHOD: Here, we delivered stimulation to the visual cortex via a high-channel-count prosthesis while recording neuronal activity and stimulation artifacts. We quantified the waveforms and temporal properties of stimulation artifacts from the cortical visual prosthesis (CVP) and used them to build a dataset, in which we simulated the neuronal activity and the stimulation artifacts. We illustrate how to use the simulated data to evaluate the performance of six software-based artifact removal methods (Template subtraction, Linear interpolation, Polynomial fitting, Exponential fitting, SALPA and ERAASR) in a CVP application scenario. RESULTS: We here focused on stimulation artifacts caused by electrical stimulation through a high-channel-count cortical prosthesis device. We find that the Polynomial fitting and Exponential fitting methods outperform the other methods in recovering spikes and multi-unit activity. Linear interpolation and Template subtraction recovered the local-field potentials. CONCLUSION: Polynomial fitting and Exponential fitting provided a good trade-off between the quality of the recovery of spikes and multi-unit activity (MUA) and the computational complexity for a cortical prosthesis.

Recurrent neural networks that learn multi-step visual routines with reinforcement learning.

Many cognitive problems can be decomposed into series of subproblems that are solved sequentially by the brain. When subproblems are solved, relevant intermediate results need to be stored by neurons and propagated to the next subproblem, until the overarching goal has been completed. We will here consider visual tasks, which can be decomposed into sequences of elemental visual operations. Experimental evidence suggests that intermediate results of the elemental operations are stored in working memory as an enhancement of neural activity in the visual cortex. The focus of enhanced activity is then available for subsequent operations to act upon. The main question at stake is how the elemental operations and their sequencing can emerge in neural networks that are trained with only rewards, in a reinforcement learning setting. We here propose a new recurrent neural network architecture that can learn composite visual tasks that require the application of successive elemental operations. Specifically, we selected three tasks for which electrophysiological recordings of monkeys' visual cortex are available. To train the networks, we used RELEARNN, a biologically plausible four-factor Hebbian learning rule, which is local both in time and space. We report that networks learn elemental operations, such as contour grouping and visual search, and execute sequences of operations, solely based on the characteristics of the visual stimuli and the reward structure of a task. After training was completed, the activity of the units of the neural network elicited by behaviorally relevant image items was stronger than that elicited by irrelevant ones, just as has been observed in the visual cortex of monkeys solving the same tasks. Relevant information that needed to be exchanged between subroutines was maintained as a focus of enhanced activity and passed on to the subsequent subroutines. Our results demonstrate how a biologically plausible learning rule can train a recurrent neural network on multistep visual tasks.

An integrative, multiscale view on neural theories of consciousness.

How is conscious experience related to material brain processes? A variety of theories aiming to answer this age-old question have emerged from the recent surge in consciousness research, and some are now hotly debated. Although most researchers have so far focused on the development and validation of their preferred theory in relative isolation, this article, written by a group of scientists representing different theories, takes an alternative approach. Noting that various theories often try to explain different aspects or mechanistic levels of consciousness, we argue that the theories do not necessarily contradict each other. Instead, several of them may converge on fundamental neuronal mechanisms and be partly compatible and complementary, so that multiple theories can simultaneously contribute to our understanding. Here, we consider unifying, integration-oriented approaches that have so far been largely neglected, seeking to combine valuable elements from various theories.

Flexible Polymer Electrodes for Stable Prosthetic Visual Perception in Mice.

Brain interfaces that can stimulate neurons, cause minimal damage, and work for a long time will be central for future neuroprosthetics. Here, the long-term performance of highly flexible, thin polyimide shanks with several small (<15 µm) electrodes during electrical microstimulation of the visual cortex, is reported. The electrodes exhibit a remarkable stability when several billions of electrical pulses are applied in vitro. When the devices are implanted in the primary visual cortex (area V1) of mice and the animals are trained to detect electrical microstimulation, it is found that the perceptual thresholds are 2-20 microamperes (µA), which is far below the maximal currents that the electrodes can withstand. The long-term functionality of the devices in vivo is excellent, with stable performance for up to more than a year and little damage to the brain tissue. These results demonstrate the potential of thin floating electrodes for the long-term restoration of lost sensory functions.

Experience shapes chandelier cell function and structure in the visual cortex.

Detailed characterization of interneuron types in primary visual cortex (V1) has greatly contributed to understanding visual perception, yet the role of chandelier cells (ChCs) in visual processing remains poorly characterized. Using viral tracing we found that V1 ChCs predominantly receive monosynaptic input from local layer 5 pyramidal cells and higher-order cortical regions. Two-photon calcium imaging and convolutional neural network modeling revealed that ChCs are visually responsive but weakly selective for stimulus content. In mice running in a virtual tunnel, ChCs respond strongly to events known to elicit arousal, including locomotion and visuomotor mismatch. Repeated exposure of the mice to the virtual tunnel was accompanied by reduced visual responses of ChCs and structural plasticity of ChC boutons and axon initial segment length. Finally, ChCs only weakly inhibited pyramidal cells. These findings suggest that ChCs provide an arousal-related signal to layer 2/3 pyramidal cells that may modulate their activity and/or gate plasticity of their axon initial segments during behaviorally relevant events.

The representation of occluded image regions in area V1 of monkeys and humans.

Neuronal activity in the primary visual cortex (V1) is driven by feedforward input from within the neurons' receptive fields (RFs) and modulated by contextual information in regions surrounding the RF. The effect of contextual information on spiking activity occurs rapidly and is therefore challenging to dissociate from feedforward input. To address this challenge, we recorded the spiking activity of V1 neurons in monkeys viewing either natural scenes or scenes where the information in the RF was occluded, effectively removing the feedforward input. We found that V1 neurons responded rapidly and selectively to occluded scenes. V1 responses elicited by occluded stimuli could be used to decode individual scenes and could be predicted from those elicited by non-occluded images, indicating that there is an overlap between visually driven and contextual responses. We used representational similarity analysis to show that the structure of V1 representations of occluded scenes measured with electrophysiology in monkeys correlates strongly with the representations of the same scenes in humans measured with functional magnetic resonance imaging (fMRI). Our results reveal that contextual influences rapidly alter V1 spiking activity in monkeys over distances of several degrees in the visual field, carry information about individual scenes, and resemble those in human V1. VIDEO ABSTRACT.

Solving the binding problem: Assemblies form when neurons enhance their firing rate-they don't need to oscillate or synchronize.

When we look at an image, its features are represented in our visual system in a highly distributed manner, calling for a mechanism that binds them into coherent object representations. There have been different proposals for the neuronal mechanisms that can mediate binding. One hypothesis is that binding is achieved by oscillations that synchronize neurons representing features of the same perceptual object. This view allows separate communication channels between different brain areas. Another hypothesis is that binding of features that are represented in different brain regions occurs when the neurons in these areas that respond to the same object simultaneously enhance their firing rate, which would correspond to directing object-based attention to these features. This review summarizes evidence in favor of and against these two hypotheses, examining the neuronal correlates of binding and assessing the time course of perceptual grouping. I conclude that enhanced neuronal firing rates bind features into coherent object representations, whereas oscillations and synchrony are unrelated to binding.

Inversion of pop-out for a distracting feature dimension in monkey visual cortex.

During visual search, it is important to reduce the interference of distracting objects in the scene. The neuronal responses elicited by the search target stimulus are typically enhanced. However, it is equally important to suppress the representations of distracting stimuli, especially if they are salient and capture attention. We trained monkeys to make an eye movement to a unique "pop-out" shape stimulus among an array of distracting stimuli. One of these distractors had a salient color that varied across trials and differed from the color of the other stimuli, causing it to also pop-out. The monkeys were able to select the pop-out shape target with high accuracy and actively avoided the pop-out color distractor. This behavioral pattern was reflected in the activity of neurons in area V4. Responses to the shape targets were enhanced, while the activity evoked by the pop-out color distractor was only briefly enhanced, directly followed by a sustained period of pronounced suppression. These behavioral and neuronal results demonstrate a cortical selection mechanism that rapidly inverts a pop-out signal to "pop-in" for an entire feature dimension thereby facilitating goal-directed visual search in the presence of salient distractors.

Contextual drive of neuronal responses in mouse V1 in the absence of feedforward input.

Neurons in the primary visual cortex (V1) respond to stimuli in their receptive field (RF), which is defined by the feedforward input from the retina. However, V1 neurons are also sensitive to contextual information outside their RF, even if the RF itself is unstimulated. Here, we examined the cortical circuits for V1 contextual responses to gray disks superimposed on different backgrounds. Contextual responses began late and were strongest in the feedback-recipient layers of V1. They differed between the three main classes of inhibitory neurons, with particularly strong contextual drive of VIP neurons, indicating a contribution of disinhibitory circuits to contextual drive. Contextual drive was strongest when the gray disk was perceived as figure, occluding its background, rather than a hole. Our results link contextual drive in V1 to perceptual organization and provide previously unknown insight into how recurrent processing shapes the response of sensory neurons to facilitate figure perception.

Purkinje Cell Activity in the Medial and Lateral Cerebellum During Suppression of Voluntary Eye Movements in Rhesus Macaques.

Volitional suppression of responses to distracting external stimuli enables us to achieve our goals. This volitional inhibition of a specific behavior is supposed to be mainly mediated by the cerebral cortex. However, recent evidence supports the involvement of the cerebellum in this process. It is currently not known whether different parts of the cerebellar cortex play differential or synergistic roles in the planning and execution of this behavior. Here, we measured Purkinje cell (PC) responses in the medial and lateral cerebellum in two rhesus macaques during pro- and anti-saccade tasks. During an antisaccade trial, non-human primates (NHPs) were instructed to make a saccadic eye movement away from a target, rather than toward it, as in prosaccade trials. Our data show that the cerebellum plays an important role not only during the execution of the saccades but also during the volitional inhibition of eye movements toward the target. Simple spike (SS) modulation during the instruction and execution periods of pro- and anti-saccades was prominent in PCs of both the medial and lateral cerebellum. However, only the SS activity in the lateral cerebellar cortex contained information about stimulus identity and showed a strong reciprocal interaction with complex spikes (CSs). Moreover, the SS activity of different PC groups modulated bidirectionally in both of regions, but the PCs that showed facilitating and suppressive activity were predominantly associated with instruction and execution, respectively. These findings show that different cerebellar regions and PC groups contribute to goal-directed behavior and volitional inhibition, but with different propensities, highlighting the rich repertoire of the cerebellar control in executive functions.

1024-channel electrophysiological recordings in macaque V1 and V4 during resting state.

Co-variations in resting state activity are thought to arise from a variety of correlated inputs to neurons, such as bottom-up activity from lower areas, feedback from higher areas, recurrent processing in local circuits, and fluctuations in neuromodulatory systems. Most studies have examined resting state activity throughout the brain using MRI scans, or observed local co-variations in activity by recording from a small number of electrodes. We carried out electrophysiological recordings from over a thousand chronically implanted electrodes in the visual cortex of non-human primates, yielding a resting state dataset with unprecedentedly high channel counts and spatiotemporal resolution. Such signals could be used to observe brain waves across larger regions of cortex, offering a temporally detailed picture of brain activity. In this paper, we provide the dataset, describe the raw and processed data formats and data acquisition methods, and indicate how the data can be used to yield new insights into the 'background' activity that influences the processing of visual information in our brain.

Population receptive fields in nonhuman primates from whole-brain fMRI and large-scale neurophysiology in visual cortex.

Population receptive field (pRF) modeling is a popular fMRI method to map the retinotopic organization of the human brain. While fMRI-based pRF maps are qualitatively similar to invasively recorded single-cell receptive fields in animals, it remains unclear what neuronal signal they represent. We addressed this question in awake nonhuman primates comparing whole-brain fMRI and large-scale neurophysiological recordings in areas V1 and V4 of the visual cortex. We examined the fits of several pRF models based on the fMRI blood-oxygen-level-dependent (BOLD) signal, multi-unit spiking activity (MUA), and local field potential (LFP) power in different frequency bands. We found that pRFs derived from BOLD-fMRI were most similar to MUA-pRFs in V1 and V4, while pRFs based on LFP gamma power also gave a good approximation. fMRI-based pRFs thus reliably reflect neuronal receptive field properties in the primate brain. In addition to our results in V1 and V4, the whole-brain fMRI measurements revealed retinotopic tuning in many other cortical and subcortical areas with a consistent increase in pRF size with increasing eccentricity, as well as a retinotopically specific deactivation of default mode network nodes similar to previous observations in humans.

A neuronal basis of iconic memory in macaque primary visual cortex.

After a briefly presented visual stimulus disappears, observers retain a detailed representation of this stimulus for a short period of time. This sensory storage is called iconic memory. We measured iconic memory in the perception of monkeys and its neuronal correlates in the primary visual cortex (area V1). We determined how many milliseconds extra viewing time iconic memory is worth and how it decays by varying the duration of a brief stimulus and the timing of a mask. The V1 activity that persists after the disappearance of a stimulus predicted accuracy, with a time course resembling the worth and decay of iconic memory. Finally, we examined how iconic memory interacts with attention. A cue presented after the stimulus disappears boosts attentional influences pertaining to a relevant part of the stimulus but only if it appears before iconic memory decayed. Our results relate iconic memory to neuronal activity in early visual cortex.

Visual percepts evoked with an intracortical 96-channel microelectrode array inserted in human occipital cortex.

BACKGROUNDA long-held goal of vision therapy is to transfer information directly to the visual cortex of blind individuals, thereby restoring a rudimentary form of sight. However, no clinically available cortical visual prosthesis yet exists.METHODSWe implanted an intracortical microelectrode array consisting of 96 electrodes in the visual cortex of a 57-year-old person with complete blindness for a 6-month period. We measured thresholds and the characteristics of the visual percepts elicited by intracortical microstimulation.RESULTSImplantation and subsequent explantation of intracortical microelectrodes were carried out without complications. The mean stimulation threshold for single electrodes was 66.8 ± 36.5 µA. We consistently obtained high-quality recordings from visually deprived neurons and the stimulation parameters remained stable over time. Simultaneous stimulation via multiple electrodes was associated with a significant reduction in thresholds (P < 0.001, ANOVA) and evoked discriminable phosphene percepts, allowing the blind participant to identify some letters and recognize object boundaries.CONCLUSIONSOur results demonstrate the safety and efficacy of chronic intracortical microstimulation via a large number of electrodes in human visual cortex, showing its high potential for restoring functional vision in the blind.TRIAL REGISTRATIONClinicalTrials.gov identifier NCT02983370.FUNDINGThe Spanish Ministerio de Ciencia Innovación y Universidades, the Generalitat Valenciana (Spain), the Europan Union's Horizon 2020 programme, the Bidons Egara Research Chair of the University Miguel Hernández (Spain), and the John Moran Eye Center of the University of Utah.

Theta-phase dependent neuronal coding during sequence learning in human single neurons.

The ability to maintain a sequence of items in memory is a fundamental cognitive function. In the rodent hippocampus, the representation of sequentially organized spatial locations is reflected by the phase of action potentials relative to the theta oscillation (phase precession). We investigated whether the timing of neuronal activity relative to the theta brain oscillation also reflects sequence order in the medial temporal lobe of humans. We used a task in which human participants learned a fixed sequence of pictures and recorded single neuron and local field potential activity with implanted electrodes. We report that spikes for three consecutive items in the sequence (the preferred stimulus for each cell, as well as the stimuli immediately preceding and following it) were phase-locked at distinct phases of the theta oscillation. Consistent with phase precession, spikes were fired at progressively earlier phases as the sequence advanced. These findings generalize previous findings in the rodent hippocampus to the human temporal lobe and suggest that encoding stimulus information at distinct oscillatory phases may play a role in maintaining sequential order in memory.

A Direct Comparison of Spatial Attention and Stimulus-Response Compatibility between Mice and Humans.

Mice are becoming an increasingly popular model for investigating the neural substrates of visual processing and higher cognitive functions. To validate the translation of mouse visual attention and sensorimotor processing to humans, we compared their performance in the same visual task. Mice and human participants judged the orientation of a grating presented on either the right or left side in the visual field. To induce shifts of spatial attention, we varied the stimulus probability on each side. As expected, human participants showed faster RTs and a higher accuracy for the side with a higher probability, a well-established effect of visual attention. The attentional effect was only present in mice when their response was slow. Although the task demanded a judgment of grating orientation, the accuracy of the mice was strongly affected by whether the side of the stimulus corresponded to the side of the behavioral response. This stimulus-response compatibility (Simon) effect was much weaker in humans and only significant for their fastest responses. Both species exhibited a speed-accuracy trade-off in their responses, because slower responses were more accurate than faster responses. We found that mice typically respond very fast, which contributes to the stronger stimulus-response compatibility and weaker attentional effects, which were only apparent in the trials with slowest responses. Humans responded slower and had stronger attentional effects, combined with a weak influence of stimulus-response compatibility, which was only apparent in trials with fast responses. We conclude that spatial attention and stimulus-response compatibility influence the responses of humans and mice but that strategy differences between species determine the dominance of these effects.

The essential role of recurrent processing for figure-ground perception in mice.

The segregation of figures from the background is an important step in visual perception. In primary visual cortex, figures evoke stronger activity than backgrounds during a delayed phase of the neuronal responses, but it is unknown how this figure-ground modulation (FGM) arises and whether it is necessary for perception. Here, we show, using optogenetic silencing in mice, that the delayed V1 response phase is necessary for figure-ground segregation. Neurons in higher visual areas also exhibit FGM and optogenetic silencing of higher areas reduced FGM in V1. In V1, figures elicited higher activity of vasoactive intestinal peptide-expressing (VIP) interneurons than the background, whereas figures suppressed somatostatin-positive interneurons, resulting in an increased activation of pyramidal cells. Optogenetic silencing of VIP neurons reduced FGM in V1, indicating that disinhibitory circuits contribute to FGM. Our results provide insight into how lower and higher areas of the visual cortex interact to shape visual perception.

Mouse visual cortex contains a region of enhanced spatial resolution.

The representation of space in mouse visual cortex was thought to be relatively uniform. Here we reveal, using population receptive-field (pRF) mapping techniques, that mouse visual cortex contains a region in which pRFs are considerably smaller. This region, the "focea," represents a location in space in front of, and slightly above, the mouse. Using two-photon imaging we show that the smaller pRFs are due to lower scatter of receptive-fields at the focea and an over-representation of binocular regions of space. We show that receptive-fields of single-neurons in areas LM and AL are smaller at the focea and that mice have improved visual resolution in this region of space. Furthermore, freely moving mice make compensatory eye-movements to hold this region in front of them. Our results indicate that mice have spatial biases in their visual processing, a finding that has important implications for the use of the mouse model of vision.

Flexible Working Memory Through Selective Gating and Attentional Tagging.

Working memory is essential: it serves to guide intelligent behavior of humans and nonhuman primates when task-relevant stimuli are no longer present to the senses. Moreover, complex tasks often require that multiple working memory representations can be flexibly and independently maintained, prioritized, and updated according to changing task demands. Thus far, neural network models of working memory have been unable to offer an integrative account of how such control mechanisms can be acquired in a biologically plausible manner. Here, we present WorkMATe, a neural network architecture that models cognitive control over working memory content and learns the appropriate control operations needed to solve complex working memory tasks. Key components of the model include a gated memory circuit that is controlled by internal actions, encoding sensory information through untrained connections, and a neural circuit that matches sensory inputs to memory content. The network is trained by means of a biologically plausible reinforcement learning rule that relies on attentional feedback and reward prediction errors to guide synaptic updates. We demonstrate that the model successfully acquires policies to solve classical working memory tasks, such as delayed recognition and delayed pro-saccade/anti-saccade tasks. In addition, the model solves much more complex tasks, including the hierarchical 12-AX task or the ABAB ordered recognition task, both of which demand an agent to independently store and updated multiple items separately in memory. Furthermore, the control strategies that the model acquires for these tasks subsequently generalize to new task contexts with novel stimuli, thus bringing symbolic production rule qualities to a neural network architecture. As such, WorkMATe provides a new solution for the neural implementation of flexible memory control.

Shape perception via a high-channel-count neuroprosthesis in monkey visual cortex.

Blindness affects 40 million people across the world. A neuroprosthesis could one day restore functional vision in the blind. We implanted a 1024-channel prosthesis in areas V1 and V4 of the visual cortex of monkeys and used electrical stimulation to elicit percepts of dots of light (called phosphenes) on hundreds of electrodes, the locations of which matched the receptive fields of the stimulated neurons. Activity in area V4 predicted phosphene percepts that were elicited in V1. We simultaneously stimulated multiple electrodes to impose visible patterns composed of a number of phosphenes. The monkeys immediately recognized them as simple shapes, motions, or letters. These results demonstrate the potential of electrical stimulation to restore functional, life-enhancing vision in the blind.