Does phasic dopamine release cause policy updates?

Phasic dopamine activity is believed to both encode reward-prediction errors (RPEs) and to cause the adaptations that these errors engender. If so, a rat working for optogenetic stimulation of dopamine neurons will repeatedly update its policy and/or action values, thus iteratively increasing its work rate. Here, we challenge this view by demonstrating stable, non-maximal work rates in the face of repeated optogenetic stimulation of midbrain dopamine neurons. Furthermore, we show that rats learn to discriminate between world states distinguished only by their history of dopamine activation. Comparison of these results to reinforcement learning simulations suggests that the induced dopamine transients acted more as rewards than RPEs. However, pursuit of dopaminergic stimulation drifted upwards over a time scale of days and weeks, despite its stability within trials. To reconcile the results with prior findings, we consider multiple roles for dopamine signalling.

Reciprocal effects of single or repeated exposure to methylphenidate or sex in adult male rats.

RATIONALE: Exposure to rewards can alter behavioral reactivity to them. For example, stimulants sensitize locomotor activation, whereas sexual experience sensitizes copulatory behaviors. Moreover, rewards can cross-sensitize one another. Although stimulants are known to cross-sensitize locomotor effects, the evidence for cross-sensitization between stimulants and sex is less clear. OBJECTIVES: This study determined the effects of single and repeated pre-exposure to methylphenidate (MPH) or sex on one another in adult male rats. METHODS: Cross-sensitization between MPH (5 mg/kg) and sex (30 min with sexually experienced female) was examined. Adult male rats were pre-exposed to 0, 1, or 10 trials of either sex or MPH before being exposed to the other reward. Locomotor chambers were used in MPH trials. Bilevel chambers were used in sexual trials, and sexual behaviors were video scored. RESULTS: The amount of prior sexual experience differentially influenced the ceiling of MPH-dependent sensitization; in the last drug trial, locomotion was highest in males given 1 previous sexual trial compared with 0 or 10. Compared with MPH-naive males, pre-exposure to MPH (1 and 10 trials) reduced the number of ejaculations without impacting sexual performance (intromission/mount latency and frequency). CONCLUSIONS: These findings indicate that the degree of pre-exposure to a reward can differentially affect reactivity to novel rewards. The results showed that previous findings of cross-sensitization between amphetamine and sex do not extend to MPH. However, exposure to MPH prior to sexual experience can increase the amount of sexual stimulation needed to achieve ejaculation.

Dopamine and Beyond: Implications of Psychophysical Studies of Intracranial Self-Stimulation for the Treatment of Depression.

Major depressive disorder is a leading cause of disability and suicide worldwide. Consecutive rounds of conventional interventions are ineffective in a significant sub-group of patients whose disorder is classified as treatment-resistant depression. Significant progress in managing this severe form of depression has been achieved through the use of deep brain stimulation of the medial forebrain bundle (MFB). The beneficial effect of such stimulation appears strong, safe, and enduring. The proposed neural substrate for this promising clinical finding includes midbrain dopamine neurons and a subset of their cortical afferents. Here, we aim to broaden the discussion of the candidate circuitry by exploring potential implications of a new "convergence" model of brain reward circuitry in rodents. We chart the evolution of the new model from its predecessors, which held that midbrain dopamine neurons constituted an obligatory stage of the final common path for reward seeking. In contrast, the new model includes a directly activated, non-dopaminergic pathway whose output ultimately converges with that of the dopaminergic neurons. On the basis of the new model and the relative ineffectiveness of dopamine agonists in the treatment of depression, we ask whether non-dopaminergic circuitry may contribute to the clinical efficacy of deep brain stimulation of the MFB.

The Convergence Model of Brain Reward Circuitry: Implications for Relief of Treatment-Resistant Depression by Deep-Brain Stimulation of the Medial Forebrain Bundle.

Deep-brain stimulation of the medial forebrain bundle (MFB) can provide effective, enduring relief of treatment-resistant depression. Panksepp provided an explanatory framework: the MFB constitutes the core of the neural circuitry subserving the anticipation and pursuit of rewards: the "SEEKING" system. On that view, the SEEKING system is hypoactive in depressed individuals; background electrical stimulation of the MFB alleviates symptoms by normalizing activity. Panksepp attributed intracranial self-stimulation to excitation of the SEEKING system in which the ascending projections of midbrain dopamine neurons are an essential component. In parallel with Panksepp's qualitative work, intracranial self-stimulation has long been studied quantitatively by psychophysical means. That work argues that the predominant directly stimulated substrate for MFB self-stimulation are myelinated, non-dopaminergic fibers, more readily excited by brief electrical current pulses than the thin, unmyelinated axons of the midbrain dopamine neurons. The series-circuit hypothesis reconciles this view with the evidence implicating dopamine in MFB self-stimulation as follows: direct activation of myelinated MFB fibers is rewarding due to their trans-synaptic activation of midbrain dopamine neurons. A recent study in which rats worked for optogenetic stimulation of midbrain dopamine neurons challenges the series-circuit hypothesis and provides a new model of intracranial self-stimulation in which the myelinated non-dopaminergic neurons and the midbrain dopamine projections access the behavioral final common path for reward seeking via separate, converging routes. We explore the potential implications of this convergence model for the interpretation of the antidepressant effect of MFB stimulation. We also discuss the consistent finding that psychomotor stimulants, which boost dopaminergic neurotransmission, fail to provide a monotherapy for depression. We propose that non-dopaminergic MFB components may contribute to the therapeutic effect in parallel to, in synergy with, or even instead of, a dopaminergic component.

The trade-off between pulse duration and power in optical excitation of midbrain dopamine neurons approximates Bloch's law.

Optogenetic experiments reveal functional roles of specific neurons. However, functional inferences have been limited by widespread adoption of a restricted set of stimulation parameters. Broader exploration of the parameter space can deepen insight into the mapping between selective neural activity and behavior. In this way, characteristics of the activated neural circuit, such as temporal integration, can be inferred. Our objective was to determine whether an equal-energy principle accounts for the interaction of pulse duration and optical power in optogenetic excitation. Six male TH::Cre rats worked for optogenetic (ChannelRhodopsin-2) stimulation of VTA dopamine neurons. We used a within-subject design to describe the trade-off between pulse duration and optical power in determining reward seeking. Parameters were customized for each subject based on behavioral effectiveness. Within a useful range of powers (~12.6-31.6 mW) the product of optical power and pulse duration required to produce a given level of reward seeking was roughly constant. Such reciprocity is consistent with Bloch's law, which posits an equal-energy principle of temporal summation over short durations in human vision. The trade-off between pulse duration and power broke down at higher powers. Thus, optical power and duration can be adjusted reciprocally for brief durations and lower powers, and power can be substituted for pulse duration to scale the region of excitation in behavioral optogenetic experiments. The findings demonstrate the utility of within-subject and trade-off designs in optogenetics and of parameter adjustment based on functional endpoints instead of physical properties of the stimulation.