Extracting single-trial neural interaction using latent dynamical systems model.

In systems neuroscience, advances in simultaneous recording technology have helped reveal the population dynamics that underlie the complex neural correlates of animal behavior and cognitive processes. To investigate these correlates, neural interactions are typically abstracted from spike trains of pairs of neurons accumulated over the course of many trials. However, the resultant averaged values do not lead to understanding of neural computation in which the responses of populations are highly variable even under identical external conditions. Accordingly, neural interactions within the population also show strong fluctuations. In the present study, we introduce an analysis method reflecting the temporal variation of neural interactions, in which cross-correlograms on rate estimates are applied via a latent dynamical systems model. Using this method, we were able to predict time-varying neural interactions within a single trial. In addition, the pairwise connections estimated in our analysis increased along behavioral epochs among neurons categorized within similar functional groups. Thus, our analysis method revealed that neurons in the same groups communicate more as the population gets involved in the assigned task. We also showed that the characteristics of neural interaction from our model differ from the results of a typical model employing cross-correlation coefficients. This suggests that our model can extract nonoverlapping information about network topology, unlike the typical model.

Role of the hippocampal CA1 region in incremental value learning.

It is generally believed that the hippocampus plays a crucial role in declarative memory-remembering facts and events-but not in gradual stimulus-response association or incremental value learning. Based on the finding that CA1 conveys strong value signals during dynamic foraging, we investigated the possibility that the hippocampus contributes to incremental value learning. Specifically, we examined effects of inactivating different subregions of the dorsal hippocampus on behavioral performance of mice performing a dynamic foraging task in a modified T-maze. A reinforcement learning model-based analysis indicated that inactivation of CA1, but not dentate gyrus, CA3, or CA2, impaired trial-by-trial updating of chosen value without affecting value-dependent action selection. As a result, it took longer for CA1-inactivated mice to bias their choices toward the higher-reward-probability target after changes in reward probability. Our results indicate, contrary to the traditional view, that the hippocampus, especially CA1, might contribute to incremental value learning under certain circumstances.

Neural Signals Related to Outcome Evaluation Are Stronger in CA1 than CA3.

We have shown previously that CA1 conveys significant neural signals necessary to update value of the chosen target, namely chosen value and reward signals. To better understand hippocampal neural processes related to valuation, we compared chosen value- and reward-related neural activity between the CA3 and CA1 regions. Single units were recorded with tetrodes from the dorsal CA3 and CA1 regions of rats performing a dynamic foraging task, and chosen value- and reward-related neural activity was estimated using a reinforcement learning model and multiple regression analyses. Neural signals for chosen value and reward converged in both CA3 and CA1 when a trial outcome was revealed. However, these neural signals were stronger in CA1 than CA3. Consequently, neural signals for reward prediction error and updated chosen value were stronger in CA1 than CA3. Together with our previous finding that CA1 conveys stronger value signals than the subiculum, our results raise the possibility that CA1 might play a particularly important role among hippocampal subregions in evaluating experienced events.

Neuronal activity in dorsomedial and dorsolateral striatum under the requirement for temporal credit assignment.

To investigate neural processes underlying temporal credit assignment in the striatum, we recorded neuronal activity in the dorsomedial and dorsolateral striatum (DMS and DLS, respectively) of rats performing a dynamic foraging task in which a choice has to be remembered until its outcome is revealed for correct credit assignment. Choice signals appeared sequentially, initially in the DMS and then in the DLS, and they were combined with action value and reward signals in the DLS when choice outcome was revealed. Unlike in conventional dynamic foraging tasks, neural signals for chosen value were elevated in neither brain structure. These results suggest that dynamics of striatal neural signals related to evaluating choice outcome might differ drastically depending on the requirement for temporal credit assignment. In a behavioral context requiring temporal credit assignment, the DLS, but not the DMS, might be in charge of updating the value of chosen action by integrating choice, action value, and reward signals together.

Effects of fictive reward on rat's choice behavior.

Choices of humans and non-human primates are influenced by both actually experienced and fictive outcomes. To test whether this is also the case in rodents, we examined rat's choice behavior in a binary choice task in which variable magnitudes of actual and fictive rewards were delivered. We found that the animal's choice was significantly influenced by the magnitudes of both actual and fictive rewards in the previous trial. A model-based analysis revealed, however, that the effect of fictive reward was more transient and influenced mostly the choice in the next trial, whereas the effect of actual reward was more sustained, consistent with incremental learning of action values. Our results suggest that the capacity to modify future choices based on fictive outcomes might be shared by many different animal species, but fictive outcomes are less effective than actual outcomes in the incremental value learning system.

Role of dopamine D2 receptors in optimizing choice strategy in a dynamic and uncertain environment.

In order to investigate roles of dopamine receptor subtypes in reward-based learning, we examined choice behavior of dopamine D1 and D2 receptor-knockout (D1R-KO and D2R-KO, respectively) mice in an instrumental learning task with progressively increasing reversal frequency and a dynamic two-armed bandit task. Performance of D2R-KO mice was progressively impaired in the former as the frequency of reversal increased and profoundly impaired in the latter even with prolonged training, whereas D1R-KO mice showed relatively minor performance deficits. Choice behavior in the dynamic two-armed bandit task was well explained by a hybrid model including win-stay-lose-switch and reinforcement learning terms. A model-based analysis revealed increased win-stay, but impaired value updating and decreased value-dependent action selection in D2R-KO mice, which were detrimental to maximizing rewards in the dynamic two-armed bandit task. These results suggest an important role of dopamine D2 receptors in learning from past choice outcomes for rapid adjustment of choice behavior in a dynamic and uncertain environment.

Functional relationships between the hippocampus and dorsomedial striatum in learning a visual scene-based memory task in rats.

The hippocampus is important for contextual behavior, and the striatum plays key roles in decision making. When studying the functional relationships with the hippocampus, prior studies have focused mostly on the dorsolateral striatum (DLS), emphasizing the antagonistic relationships between the hippocampus and DLS in spatial versus response learning. By contrast, the functional relationships between the dorsomedial striatum (DMS) and hippocampus are relatively unknown. The current study reports that lesions to both the hippocampus and DMS profoundly impaired performance of rats in a visual scene-based memory task in which the animals were required to make a choice response by using visual scenes displayed in the background. Analysis of simultaneous recordings of local field potentials revealed that the gamma oscillatory power was higher in the DMS, but not in CA1, when the rat performed the task using familiar scenes than novel ones. In addition, the CA1-DMS networks increased coherence at γ, but not at θ, rhythm as the rat mastered the task. At the single-unit level, the neuronal populations in CA1 and DMS showed differential firing patterns when responses were made using familiar visual scenes than novel ones. Such learning-dependent firing patterns were observed earlier in the DMS than in CA1 before the rat made choice responses. The present findings suggest that both the hippocampus and DMS process memory representations for visual scenes in parallel with different time courses and that flexible choice action using background visual scenes requires coordinated operations of the hippocampus and DMS at γ frequencies.

Human serum transthyretin levels correlate inversely with Alzheimer's disease.

Alzheimer's disease (AD) is the fastest growing neurodegenerative disease in the elderly population, and the search for therapeutic targets and diagnostic AD biomarkers is an exigent issue. Because amyloid-ß (Aß) aggregation constitutes the epicenter of AD pathology, Aß-binding proteins that regulate Aß aggregation, such as transthyretin (TTR), have attracted much attention. TTR binds to Aß, prevents its aggregation, and consequently inhibits Aß-induced cellular toxicity. Decreased TTR levels in cerebrospinal fluid (CSF) from AD patients suggest that TTR is a biomarker of AD. But, studies on TTR as a biomarker have focused on CSF; no study has evaluated peripheral levels of TTR in AD. Here, we examined the relationship between serum TTR levels and AD. We measured TTR levels in serum samples from 90 nondemented controls and 111 AD patients and observed significantly lower serum TTR levels in AD (p < 0.001). Notably, females in the control group had lower serum TTR levels compared with male in the control (p = 0.006), while no difference in gender was noted in the AD group. There were no age-related changes in serum TTR levels. Thus, this study demonstrates a clear negative correlation between serum TTR levels and AD, suggesting that TTR is not only involved in AD pathological process but also suggested as possible peripheral biomarker for AD diagnosis in serum level.

Distinct roles of rodent orbitofrontal and medial prefrontal cortex in decision making.

We investigated how different subregions of rodent prefrontal cortex contribute to value-based decision making, by comparing neural signals related to animal's choice, its outcome, and action value in orbitofrontal cortex (OFC) and medial prefrontal cortex (mPFC) of rats performing a dynamic two-armed bandit task. Neural signals for upcoming action selection arose in the mPFC, including the anterior cingulate cortex, only immediately before the behavioral manifestation of animal's choice, suggesting that rodent prefrontal cortex is not involved in advanced action planning. Both OFC and mPFC conveyed signals related to the animal's past choices and their outcomes over multiple trials, but neural signals for chosen value and reward prediction error were more prevalent in the OFC. Our results suggest that rodent OFC and mPFC serve distinct roles in value-based decision making and that the OFC plays a prominent role in updating the values of outcomes expected from chosen actions.

Role of striatum in updating values of chosen actions.

The striatum is thought to play a crucial role in value-based decision making. Although a large body of evidence suggests its involvement in action selection as well as action evaluation, underlying neural processes for these functions of the striatum are largely unknown. To obtain insights on this matter, we simultaneously recorded neuronal activity in the dorsal and ventral striatum of rats performing a dynamic two-armed bandit task, and examined temporal profiles of neural signals related to animal's choice, its outcome, and action value. Whereas significant neural signals for action value were found in both structures before animal's choice of action, signals related to the upcoming choice were relatively weak and began to emerge only in the dorsal striatum approximately 200 ms before the behavioral manifestation of the animal's choice. In contrast, once the animal revealed its choice, signals related to choice and its value increased steeply and persisted until the outcome of animal's choice was revealed, so that some neurons in both structures concurrently conveyed signals related to animal's choice, its outcome, and the value of chosen action. Thus, all the components necessary for updating values of chosen actions were available in the striatum. These results suggest that the striatum not only represents values associated with potential choices before animal's choice of action, but might also update the value of chosen action once its outcome is revealed. In contrast, action selection might take place elsewhere or in the dorsal striatum only immediately before its behavioral manifestation.

Model-based reinforcement learning under concurrent schedules of reinforcement in rodents.

Reinforcement learning theories postulate that actions are chosen to maximize a long-term sum of positive outcomes based on value functions, which are subjective estimates of future rewards. In simple reinforcement learning algorithms, value functions are updated only by trial-and-error, whereas they are updated according to the decision-maker's knowledge or model of the environment in model-based reinforcement learning algorithms. To investigate how animals update value functions, we trained rats under two different free-choice tasks. The reward probability of the unchosen target remained unchanged in one task, whereas it increased over time since the target was last chosen in the other task. The results show that goal choice probability increased as a function of the number of consecutive alternative choices in the latter, but not the former task, indicating that the animals were aware of time-dependent increases in arming probability and used this information in choosing goals. In addition, the choice behavior in the latter task was better accounted for by a model-based reinforcement learning algorithm. Our results show that rats adopt a decision-making process that cannot be accounted for by simple reinforcement learning models even in a relatively simple binary choice task, suggesting that rats can readily improve their decision-making strategy through the knowledge of their environments.

Information transmission by stimulus-dependent modulation of noise correlation.

Cortical neurons that are near one another show correlated response variability (noise correlation), which can contribute to synergistic information transmission. In this study, we investigated the relationship between the level of external stimulation and noise correlation and its effect on population coding. Six levels of electrical stimulation were delivered to a rat's hind paw and responses of several neighboring neurons were simultaneously recorded in the primary somatosensory cortex. As the intensity of stimulation increased, noise correlation decreased down to near zero and then increased again to a relatively small value. The degree of synergistic information transmission depended on the amount by which noise correlation was modulated. Our results show that noise correlation among somatosensory cortical neurons is dynamically modulated by external stimulation, which allows transmission of additional information.

Encoding of action history in the rat ventral striatum.

In a dynamic environment, animals need to update information about the rewards expected from their alternative actions continually to make optimal choices for its survival. Because the reward resulting from a given action can be substantially delayed, the process of linking a reward to its causative action would be facilitated by memory signals related to the animal's previous actions. Although the ventral striatum has been proposed to play a key role in updating the information about the rewards expected from specific actions, it is not known whether the signals related to previous actions exist in the ventral striatum. In the present study, we recorded neuronal ensemble activity in the rat ventral striatum during a visual discrimination task and investigated whether neuronal activity in the ventral striatum encoded signals related to animal's previous actions. The results show that many neurons modulated their activity according to the animal's goal choice in the previous trial, indicating that memory signals for previous actions are available in the ventral striatum. In contrast, few neurons conveyed signals on impending goal choice of the animal, suggesting the absence of decision signals in the ventral striatum. Memory signals for previous actions might contribute to the process of updating the estimates of rewards expected from alternative actions in the ventral striatum.