Time-varying generalized linear models: characterizing and decoding neuronal dynamics in higher visual areas.

To create a behaviorally relevant representation of the visual world, neurons in higher visual areas exhibit dynamic response changes to account for the time-varying interactions between external (e.g., visual input) and internal (e.g., reward value) factors. The resulting high-dimensional representational space poses challenges for precisely quantifying individual factors' contributions to the representation and readout of sensory information during a behavior. The widely used point process generalized linear model (GLM) approach provides a powerful framework for a quantitative description of neuronal processing as a function of various sensory and non-sensory inputs (encoding) as well as linking particular response components to particular behaviors (decoding), at the level of single trials and individual neurons. However, most existing variations of GLMs assume the neural systems to be time-invariant, making them inadequate for modeling nonstationary characteristics of neuronal sensitivity in higher visual areas. In this review, we summarize some of the existing GLM variations, with a focus on time-varying extensions. We highlight their applications to understanding neural representations in higher visual areas and decoding transient neuronal sensitivity as well as linking physiology to behavior through manipulation of model components. This time-varying class of statistical models provide valuable insights into the neural basis of various visual behaviors in higher visual areas and hold significant potential for uncovering the fundamental computational principles that govern neuronal processing underlying various behaviors in different regions of the brain.

Prefrontal activity sharpens spatial sensitivity of extrastriate neurons.

Prefrontal cortex is known to exert its control over representation of visual signals in extrastriate areas such as V4. Frontal Eye Field (FEF) is suggested to be the proxy for the prefrontal control of visual signals. However, it is not known which aspects of sensory representation within extrastriate areas are under the influence of FEF activity. We employed a causal manipulation to examine how FEF activity contributes to spatial sensitivity of extrastriate neurons. Finding FEF and V4 areas with overlapping response field (RF) in two macaque monkeys, we recorded V4 responses before and after inactivation of the overlapping FEF. We assessed spatial sensitivity of V4 neurons in terms of their response gain, RF spread, coding capacity, and spatial discriminability. Unexpectedly, we found that in the absence of FEF activity, spontaneous and visually-evoked activity of V4 neurons both increase and their RFs enlarge. However, assessing the spatial sensitivity within V4, we found that these changes were associated with a reduction in the ability of V4 neurons to represent spatial information: After FEF inactivation, V4 neurons showed a reduced response gain and a decrease in their spatial discriminability and coding capacity. These results show the necessity of FEF activity for shaping spatial responses of extrastriate neurons and indicates the importance of FEF inputs in sharpening the sensitivity of V4 responses.

Neural correlates of perisaccadic visual mislocalization in extrastriate cortex.

When interacting with the visual world using saccadic eye movements (saccades), the perceived location of visual stimuli becomes biased, a phenomenon called perisaccadic mislocalization, which is indeed an exemplar of the brain's dynamic representation of the visual world. However, the neural mechanism underlying this altered visuospatial perception and its potential link to other perisaccadic perceptual phenomena have not been established. Using a combined experimental and computational approach, we were able to quantify spatial bias around the saccade target (ST) based on the perisaccadic dynamics of extrastriate spatiotemporal sensitivity captured by statistical models. This approach could predict the perisaccadic spatial bias around the ST, consistent with the psychophysical studies, and revealed the precise neuronal response components underlying representational bias. These findings also established the crucial role of response remapping toward ST representation for neurons with receptive fields far from the ST in driving the ST spatial bias. Moreover, we showed that, by allocating more resources for visual target representation, visual areas enhance their representation of the ST location, even at the expense of transient distortions in spatial representation. This potential neural basis for perisaccadic ST representation, also supports a general role for extrastriate neurons in creating the perception of stimulus location.

Conductive Polymer Enabled Biostable Liquid Metal Electrodes for Bioelectronic Applications.

Gallium (Ga)-based liquid metal materials have emerged as a promising material platform for soft bioelectronics. Unfortunately, Ga has limited biostability and electrochemical performance under physiological conditions, which can hinder the implementation of its use in bioelectronic devices. Here, an effective conductive polymer deposition strategy on the liquid metal surface to improve the biostability and electrochemical performance of Ga-based liquid metals for use under physiological conditions is demonstrated. The conductive polymer [poly(3,4-ethylene dioxythiophene):tetrafluoroborate]-modified liquid metal surface significantly outperforms the liquid metal.based electrode in mechanical, biological, and electrochemical studies. In vivo action potential recordings in behaving nonhuman primate and invertebrate models demonstrate the feasibility of using liquid metal electrodes for high-performance neural recording applications. This is the first demonstration of single-unit neural recording using Ga-based liquid metal bioelectronic devices to date. The results determine that the electrochemical deposition of conductive polymer over liquid metal can improve the material properties of liquid metal electrodes for use under physiological conditions and open numerous design opportunities for next-generation liquid metal-based bioelectronics.

A sensory memory to preserve visual representations across eye movements.

Saccadic eye movements (saccades) disrupt the continuous flow of visual information, yet our perception of the visual world remains uninterrupted. Here we assess the representation of the visual scene across saccades from single-trial spike trains of extrastriate visual areas, using a combined electrophysiology and statistical modeling approach. Using a model-based decoder we generate a high temporal resolution readout of visual information, and identify the specific changes in neurons' spatiotemporal sensitivity that underly an integrated perisaccadic representation of visual space. Our results show that by maintaining a memory of the visual scene, extrastriate neurons produce an uninterrupted representation of the visual world. Extrastriate neurons exhibit a late response enhancement close to the time of saccade onset, which preserves the latest pre-saccadic information until the post-saccadic flow of retinal information resumes. These results show how our brain exploits available information to maintain a representation of the scene while visual inputs are disrupted.

Characterizing and dissociating multiple time-varying modulatory computations influencing neuronal activity.

In many brain areas, sensory responses are heavily modulated by factors including attentional state, context, reward history, motor preparation, learned associations, and other cognitive variables. Modelling the effect of these modulatory factors on sensory responses has proven challenging, mostly due to the time-varying and nonlinear nature of the underlying computations. Here we present a computational model capable of capturing and dissociating multiple time-varying modulatory effects on neuronal responses on the order of milliseconds. The model's performance is tested on extrastriate perisaccadic visual responses in nonhuman primates. Visual neurons respond to stimuli presented around the time of saccades differently than during fixation. These perisaccadic changes include sensitivity to the stimuli presented at locations outside the neuron's receptive field, which suggests a contribution of multiple sources to perisaccadic response generation. Current computational approaches cannot quantitatively characterize the contribution of each modulatory source in response generation, mainly due to the very short timescale on which the saccade takes place. In this study, we use a high spatiotemporal resolution experimental paradigm along with a novel extension of the generalized linear model framework (GLM), termed the sparse-variable GLM, to allow for time-varying model parameters representing the temporal evolution of the system with a resolution on the order of milliseconds. We used this model framework to precisely map the temporal evolution of the spatiotemporal receptive field of visual neurons in the middle temporal area during the execution of a saccade. Moreover, an extended model based on a factorization of the sparse-variable GLM allowed us to disassociate and quantify the contribution of individual sources to the perisaccadic response. Our results show that our novel framework can precisely capture the changes in sensitivity of neurons around the time of saccades, and provide a general framework to quantitatively track the role of multiple modulatory sources over time.

Developing a Nonstationary Computational Framework With Application to Modeling Dynamic Modulations in Neural Spiking Responses.

OBJECTIVE: This paper aims to develop a computational model that incorporates the functional effects of modulatory covariates (such as context, task, or behavior), which dynamically alter the relationship between the stimulus and the neural response. METHODS: We develop a general computational approach along with an efficient estimation procedure in the widely used generalized linear model (GLM) framework to characterize such nonstationary dynamics in spiking response and spatiotemporal characteristics of a neuron at the level of individual trials. The model employs a set of modulatory components, which nonlinearly interact with other stimulus-related signals to reproduce such nonstationary effects. RESULTS: The model is tested for its ability to predict the responses of neurons in the middle temporal cortex of macaque monkeys during an eye movement task. The fitted model proves successful in capturing the fast temporal modulations in the response, reproducing the spike response temporal statistics, and accurately accounting for the neurons' dynamic spatiotemporal sensitivities, during eye movements. CONCLUSION: The nonstationary GLM framework developed in this study can be used in cases where a time-varying behavioral or cognitive component makes GLM-based models insufficient to describe the dependencies of neural responses on the stimulus-related covariates. SIGNIFICANCE: In addition to being quite powerful in encoding time-varying response modulations, this general framework also enables a readout of the neural code while dissociating the influence of other nonstimulus covariates. This framework will advance our ability to understand sensory processing in higher brain areas when modulated by several behavioral or cognitive variables.