Causal evidence supporting the proposal that dopamine transients function as temporal difference prediction errors.

Reward-evoked dopamine transients are well established as prediction errors. However, the central tenet of temporal difference accounts-that similar transients evoked by reward-predictive cues also function as errors-remains untested. In the present communication we addressed this by showing that optogenetically shunting dopamine activity at the start of a reward-predicting cue prevents second-order conditioning without affecting blocking. These results indicate that cue-evoked transients function as temporal-difference prediction errors rather than reward predictions.

Brief, But Not Prolonged, Pauses in the Firing of Midbrain Dopamine Neurons Are Sufficient to Produce a Conditioned Inhibitor.

Prediction errors are critical for associative learning. In the brain, these errors are thought to be signaled, in part, by midbrain dopamine neurons. However, although there is substantial direct evidence that brief increases in the firing of these neurons can mimic positive prediction errors, there is less evidence that brief pauses mimic negative errors. Whereas pauses in the firing of midbrain dopamine neurons can substitute for missing negative prediction errors to drive extinction, it has been suggested that this effect might be attributable to changes in salience rather than the operation of this signal as a negative prediction error. Here we address this concern by showing that the same pattern of inhibition will create a cue able to meet the classic definition of a conditioned inhibitor by showing suppression of responding in a summation test and slower learning in a retardation test. Importantly, these classic criteria were designed to rule out explanations founded on attention or salience; thus the results cannot be explained in this manner. We also show that this pattern of behavior is not produced by a single, prolonged, ramped period of inhibition, suggesting that it is precisely timed, sudden change and not duration that conveys the teaching signal.SIGNIFICANCE STATEMENT Here we show that brief pauses in the firing of midbrain dopamine neurons are sufficient to produce a cue that meets the classic criteria defining a conditioned inhibitor, or a cue that predicts the omission of a reward. These criteria were developed to distinguish actual learning from salience or attentional effects; thus these results formally show that brief pauses in the firing of dopamine neurons can serve as key teaching signals in the brain. Interestingly, this was not true for gradual prolonged pauses, suggesting it is the dynamic change in firing that serves as the teaching signal.

Author Correction: Dopamine transients are sufficient and necessary for acquisition of model-based associations.

In the version of this article initially published, the laser activation at the start of cue X in experiment 1 was described in the first paragraph of the Results and in the third paragraph of the Experiment 1 section of the Methods as lasting 2 s; in fact, it lasted only 1 s. The error has been corrected in the HTML and PDF versions of the article.

Optogenetic Blockade of Dopamine Transients Prevents Learning Induced by Changes in Reward Features.

Prediction errors are critical for associative learning [1, 2]. Transient changes in dopamine neuron activity correlate with positive and negative reward prediction errors and can mimic their effects [3-15]. However, although causal studies show that dopamine transients of 1-2 s are sufficient to drive learning about reward, these studies do not address whether they are necessary (but see [11]). Further, the precise nature of this signal is not yet fully established. Although it has been equated with the cached-value error signal proposed to support model-free reinforcement learning, cached-value errors are typically confounded with errors in the prediction of reward features [16]. Here, we used optogenetic and transgenic approaches to prevent transient changes in midbrain dopamine neuron activity during the critical error-signaling period of two unblocking tasks. In one, learning was unblocked by increasing the number of rewards, a manipulation that induces errors in predicting both value and reward features. In another, learning was unblocked by switching from one to another equally valued reward, a manipulation that induces errors only in reward feature prediction. Preventing dopamine neurons in the ventral tegmental area from firing for 5 s beginning before and continuing until after the changes in reward prevented unblocking of learning in both tasks. A similar duration suppression did not induce extinction when delivered during an expected reward, indicating that it did not act independently as a negative prediction error. This result suggests that dopamine transients play a general role in error signaling rather than being restricted to only signaling errors in value.

Dopamine transients are sufficient and necessary for acquisition of model-based associations.

Associative learning is driven by prediction errors. Dopamine transients correlate with these errors, which current interpretations limit to endowing cues with a scalar quantity reflecting the value of future rewards. We tested whether dopamine might act more broadly to support learning of an associative model of the environment. Using sensory preconditioning, we show that prediction errors underlying stimulus-stimulus learning can be blocked behaviorally and reinstated by optogenetically activating dopamine neurons. We further show that suppressing the firing of these neurons across the transition prevents normal stimulus-stimulus learning. These results establish that the acquisition of model-based information about transitions between nonrewarding events is also driven by prediction errors and that, contrary to existing canon, dopamine transients are both sufficient and necessary to support this type of learning. Our findings open new possibilities for how these biological signals might support associative learning in the mammalian brain in these and other contexts.

Thinking Outside the Box: Orbitofrontal Cortex, Imagination, and How We Can Treat Addiction.

Addiction involves an inability to control drug-seeking behavior. While this may be thought of as secondary to an overwhelming desire for drugs, it could equally well reflect a failure of the brain mechanisms that allow addicts to learn about and mentally simulate non-drug consequences. Importantly, this process of mental simulation draws upon, but is not normally bound by, our past experiences. Rather we have the ability to think outside the box of our past, integrating knowledge gained from a variety of similar and not-so-similar life experiences to derive estimates or imagine what might happen next. These estimates influence our current behavior directly and also affect future behavior by serving as the background against which outcomes are evaluated to support learning. Here we will review evidence, from our own work using a Pavlovian over-expectation task as well as from other sources, that the orbitofrontal cortex is a critical node in the neural circuit that generates these estimates. Further we will offer the specific hypothesis that degradation of this function secondary to drug-induced changes is a critical and likely addressable part of addiction.

Brief optogenetic inhibition of dopamine neurons mimics endogenous negative reward prediction errors.

Correlative studies have strongly linked phasic changes in dopamine activity with reward prediction error signaling. But causal evidence that these brief changes in firing actually serve as error signals to drive associative learning is more tenuous. Although there is direct evidence that brief increases can substitute for positive prediction errors, there is no comparable evidence that similarly brief pauses can substitute for negative prediction errors. In the absence of such evidence, the effect of increases in firing could reflect novelty or salience, variables also correlated with dopamine activity. Here we provide evidence in support of the proposed linkage, showing in a modified Pavlovian over-expectation task that brief pauses in the firing of dopamine neurons in rat ventral tegmental area at the time of reward are sufficient to mimic the effects of endogenous negative prediction errors. These results support the proposal that brief changes in the firing of dopamine neurons serve as full-fledged bidirectional prediction error signals.

Functional and structural deficits of the dentate gyrus network coincide with emerging spontaneous seizures in an Scn1a mutant Dravet Syndrome model during development.

Dravet syndrome (DS) is characterized by severe infant-onset myoclonic epilepsy along with delayed psychomotor development and heightened premature mortality. A primary monogenic cause is mutation of the SCN1A gene, which encodes the voltage-gated sodium channel subunit Nav1.1. The nature and timing of changes caused by SCN1A mutation in the hippocampal dentate gyrus (DG) network, a core area for gating major excitatory input to hippocampus and a classic epileptogenic zone, are not well known. In particularly, it is still not clear whether the developmental deficit of this epileptogenic neural network temporally matches with the progress of seizure development. Here, we investigated the emerging functional and structural deficits of the DG network in a novel mouse model (Scn1a(E1099X/+)) that mimics the genetic deficit of human DS. Scn1a(E1099X/+) (Het) mice, similarly to human DS patients, exhibited early spontaneous seizures and were more susceptible to hyperthermia-induced seizures starting at postnatal week (PW) 3, with seizures peaking at PW4. During the same period, the Het DG exhibited a greater reduction of Nav1.1-expressing GABAergic neurons compared to other hippocampal areas. Het DG GABAergic neurons showed altered action potential kinetics, reduced excitability, and generated fewer spontaneous inhibitory inputs into DG granule cells. The effect of reduced inhibitory input to DG granule cells was exacerbated by heightened spontaneous excitatory transmission and elevated excitatory release probability in these cells. In addition to electrophysiological deficit, we observed emerging morphological abnormalities of DG granule cells. Het granule cells exhibited progressively reduced dendritic arborization and excessive spines, which coincided with imbalanced network activity and the developmental onset of spontaneous seizures. Taken together, our results establish the existence of significant structural and functional developmental deficits of the DG network and the temporal correlation between emergence of these deficits and the onset of seizures in Het animals. Most importantly, our results uncover the developmental deficits of neural connectivity in Het mice. Such structural abnormalities likely further exacerbate network instability and compromise higher-order cognitive processing later in development, and thus highlight the multifaceted impacts of Scn1a deficiency on neural development.

Orbitofrontal activation restores insight lost after cocaine use.

Addiction is characterized by a lack of insight into the likely outcomes of one's behavior. Insight, or the ability to imagine outcomes, is evident when outcomes have not been directly experienced. Using this concept, work in both rats and humans has recently identified neural correlates of insight in the medial and orbital prefrontal cortices. We found that these correlates were selectively abolished in rats by cocaine self-administration. Their abolition was associated with behavioral deficits and reduced synaptic efficacy in orbitofrontal cortex, the reversal of which by optogenetic activation restored normal behavior. These results provide a link between cocaine use and problems with insight. Deficits in these functions are likely to be particularly important for problems such as drug relapse, in which behavior fails to account for likely adverse outcomes. As such, our data provide a neural target for therapeutic approaches to address these defining long-term effects of drug use.

Neural estimates of imagined outcomes in the orbitofrontal cortex drive behavior and learning.

Imagination, defined as the ability to interpret reality in ways that diverge from past experience, is fundamental to adaptive behavior. This can be seen at a simple level in our capacity to predict novel outcomes in new situations. The ability to anticipate outcomes never before received can also influence learning if those imagined outcomes are not received. The orbitofrontal cortex is a key candidate for where the process of imagining likely outcomes occurs; however, its precise role in generating these estimates and applying them to learning remain open questions. Here we address these questions by showing that single-unit activity in the orbitofrontal cortex reflects novel outcome estimates. The strength of these neural correlates predicted both behavior and learning, learning that was abolished by temporally specific inhibition of orbitofrontal neurons. These results are consistent with the proposal that the orbitofrontal cortex is critical for integrating information to imagine future outcomes.

Calcium-independent inhibitory G-protein signaling induces persistent presynaptic muting of hippocampal synapses.

Adaptive forms of synaptic plasticity that reduce excitatory synaptic transmission in response to prolonged increases in neuronal activity may prevent runaway positive feedback in neuronal circuits. In hippocampal neurons, for example, glutamatergic presynaptic terminals are selectively silenced, creating "mute" synapses, after periods of increased neuronal activity or sustained depolarization. Previous work suggests that cAMP-dependent and proteasome-dependent mechanisms participate in silencing induction by depolarization, but upstream activators are unknown. We, therefore, tested the role of calcium and G-protein signaling in silencing induction in cultured hippocampal neurons. We found that silencing induction by depolarization was not dependent on rises in intracellular calcium, from either extracellular or intracellular sources. Silencing was, however, pertussis toxin sensitive, which suggests that inhibitory G-proteins are recruited. Surprisingly, blocking four common inhibitory G-protein-coupled receptors (GPCRs) (adenosine A(1) receptors, GABA(B) receptors, metabotropic glutamate receptors, and CB(1) cannabinoid receptors) and one ionotropic receptor with metabotropic properties (kainate receptors) failed to prevent depolarization-induced silencing. Activating a subset of these GPCRs (A(1) and GABA(B)) with agonist application induced silencing, however, which supports the hypothesis that G-protein activation is a critical step in silencing. Overall, our results suggest that depolarization activates silencing through an atypical GPCR or through receptor-independent G-protein activation. GPCR agonist-induced silencing exhibited dependence on the ubiquitin-proteasome system, as was shown previously for depolarization-induced silencing, implicating the degradation of vital synaptic proteins in silencing by GPCR activation. These data suggest that presynaptic muting in hippocampal neurons uses a G-protein-dependent but calcium-independent mechanism to depress presynaptic vesicle release.

Rapid activation of dormant presynaptic terminals by phorbol esters.

Presynaptic stimulation stochastically recruits transmission according to the release probability (P(r)) of synapses. The majority of central synapses have relatively low P(r), which includes synapses that are completely quiescent presynaptically. The presence of presynaptically dormant versus active terminals presumably increases synaptic malleability when conditions demand synaptic strengthening or weakening, perhaps by triggering second messenger signals. However, whether modulator-mediated potentiation involves recruitment of transmission from dormant terminals remains unclear. Here, by combining electrophysiological and fluorescence imaging approaches, we uncovered rapid presynaptic awakening by select synaptic modulators. A phorbol ester phorbol 12,13-dibutyrate (PDBu) (a diacylglycerol analog), but not forskolin (an adenylyl cyclase activator) or elevated extracellular calcium, recruited neurotransmission from presynaptically dormant synapses. This effect was not dependent on protein kinase C activation. After PDBu-induced awakening, these previously dormant terminals had a synaptic P(r) spectrum similar to basally active synapses naive to PDBu treatment. Dormant terminals did not seem to have properties of nascent or immature synapses, judged by NR2B NMDAR (NMDA receptor) receptor subunit contribution after PDBu-stimulated awakening. Strikingly, synapses rendered inactive by prolonged depolarization, unlike basally dormant synapses, were not awakened by PDBu. These results suggest that the initial release competence of synapses can dictate the acute response to second messenger modulation, and the results suggest multiple pathways to presynaptic dormancy and awakening.

Dynamic modulation of phasic and asynchronous glutamate release in hippocampal synapses.

Although frequency-dependent short-term presynaptic plasticity has been of long-standing interest, most studies have emphasized modulation of the synchronous, phasic component of transmitter release, most evident with a single or a few presynaptic stimuli. Asynchronous transmitter release, vesicle fusion not closely time locked to presynaptic action potentials, can also be prominent under certain conditions, including repetitive stimulation. Asynchrony has often been attributed to residual Ca(2+) buildup in the presynaptic terminal. We verified that a number of manipulations of Ca(2+) handling and influx selectively alter asynchronous release relative to phasic transmitter release during action potential trains in cultured excitatory autaptic hippocampal neurons. To determine whether other manipulations of vesicle release probability also selectively modulate asynchrony, we probed the actions of one thoroughly studied modulator class whose actions on phasic versus asynchronous release have not been investigated. We examined the effects of the phorbol ester PDBu, which has protein kinase C (PKC) dependent and independent actions on presynaptic transmitter release. PDBu increased phasic and asynchronous release in parallel. However, while PKC inhibition had relatively minor inhibitory effects on PDBu potentiation of phasic and total release during action potential trains, PKC inhibition strongly reduced phorbol-potentiated asynchrony, through actions most evident late during stimulus trains. These results lend new insight into PKC-dependent and -independent effects on transmitter release and suggest the possibility of differential control of synchronous versus asynchronous vesicle release.

Identification of the gene for vitamin K epoxide reductase.

Vitamin K epoxide reductase (VKOR) is the target of warfarin, the most widely prescribed anticoagulant for thromboembolic disorders. Although estimated to prevent twenty strokes per induced bleeding episode, warfarin is under-used because of the difficulty of controlling dosage and the fear of inducing bleeding. Although identified in 1974 (ref. 2), the enzyme has yet to be purified or its gene identified. A positional cloning approach has become possible after the mapping of warfarin resistance to rat chromosome 1 (ref. 3) and of vitamin K-dependent protein deficiencies to the syntenic region of human chromosome 16 (ref. 4). Localization of VKOR to 190 genes within human chromosome 16p12-q21 narrowed the search to 13 genes encoding candidate transmembrane proteins, and we used short interfering RNA (siRNA) pools against individual genes to test their ability to inhibit VKOR activity in human cells. Here, we report the identification of the gene for VKOR based on specific inhibition of VKOR activity by a single siRNA pool. We confirmed that MGC11276 messenger RNA encodes VKOR through its expression in insect cells and sensitivity to warfarin. The expressed enzyme is 163 amino acids long, with at least one transmembrane domain. Identification of the VKOR gene extends our understanding of blood clotting, and should facilitate development of new anticoagulant drugs.

Induction of G1 phase arrest in MCF human breast cancer cells by pentagalloylglucose through the down-regulation of CDK4 and CDK2 activities and up-regulation of the CDK inhibitors p27(Kip) and p21(Cip).

Pentagalloylglucose (5GG) is a potent and specific inhibitor of NADPH dehydrogenase or xanthine oxidase. In our previous study, we showed that 5GG was able to induce apoptosis in HL-60 cells in a time- and concentration-dependent manner via the activation of caspase-3. Recently, we found that 5GG was capable of perturbing the cell cycle of the human breast cancer cell line MCF-7. DNA flow cytometric analysis showed that 5GG exhibited the ability of blocking MCF-7 cell cycle progression at the G1 phase. The level of several G1 phase-related cyclins and cyclin-dependent kinases did not change in these cells during a 24-hr exposure to 5GG. However, the activity of cyclin E/CDK2 was decreased in a concentration- and time-dependent manner and the activity of cyclin D/CDK4 was inhibited when serum-starved synchronized cells were released from synchronization. p27(Kip) and p21(Cip), inhibitors of cyclin/CDK complexes in G1-phase, were gradually increased after 5GG treatment in a time-dependent manner and the induction of p21(Cip) was correlated with an increase in p53 levels. These results suggest that the suppression of cell-cycle progression in the G1 phase by 5GG was mediated in MCF-7 cells, at least in part, by either the inhibition of cyclin D/CDK4 and cyclin E/CDK2 activity or the induction of the CDK inhibitors p27(Kip) and p21(Cip).