Does phasic dopamine release cause policy updates?

Phasic dopamine activity is believed to both encode reward-prediction errors (RPEs) and to cause the adaptations that these errors engender. If so, a rat working for optogenetic stimulation of dopamine neurons will repeatedly update its policy and/or action values, thus iteratively increasing its work rate. Here, we challenge this view by demonstrating stable, non-maximal work rates in the face of repeated optogenetic stimulation of midbrain dopamine neurons. Furthermore, we show that rats learn to discriminate between world states distinguished only by their history of dopamine activation. Comparison of these results to reinforcement learning simulations suggests that the induced dopamine transients acted more as rewards than RPEs. However, pursuit of dopaminergic stimulation drifted upwards over a time scale of days and weeks, despite its stability within trials. To reconcile the results with prior findings, we consider multiple roles for dopamine signalling.

Longitudinal CSF proteomics identifies NPTX2 as a prognostic biomarker of Alzheimer's disease.

INTRODUCTION: Biomarkers that reflect pathologic processes affecting neuronal function during preclinical and early stages of Alzheimer's disease (AD) are needed to aid drug development. METHODS: A targeted, stable isotope, quantitative mass spectrometry-based investigation of longitudinal changes in concentrations of previously identified candidate biomarkers was performed in cerebrospinal fluid (CSF) of Alzheimer's Disease Neuroimaging Initiative participants who were classified as cognitively normal (CN; n = 76) or with mild cognitive impairment (MCI; n = 111) at baseline. RESULTS: Of the candidate biomarkers, the CSF concentration of neuronal pentraxin 2 (NPTX2), a protein involved in synaptic function, exhibited rates of change that were significantly different between three comparison groups (i.e., CN vs. MCI participants; AD pathology positive vs. negative defined by phosphorylated tau181/amyloid beta1-42 ratio; and clinical progressors vs. non-progressors). The rate of change of NPTX2 also significantly correlated with declining cognition. DISCUSSION: CSF NPTX2 concentration is a strong prognostic biomarker candidate of accelerated cognitive decline with potential use as a therapeutic target.

Discovery of a putative blood-based protein signature associated with response to ALK tyrosine kinase inhibition.

BACKGROUND: ALK tyrosine kinase inhibition has become a mainstay in the clinical management of ALK fusion positive NSCLC patients. Although ALK mutations can reliably predict the likelihood of response to ALK tyrosine kinase inhibitors (TKIs) such as crizotinib, they cannot reliably predict response duration or intrinsic/extrinsic therapeutic resistance. To further refine the application of personalized medicine in this indication, this study aimed to identify prognostic proteomic biomarkers in ALK fusion positive NSCLC patients to crizotinib. METHODS: Twenty-four patients with advanced NSCLC harboring ALK fusion were administered crizotinib in a phase IV trial which included blood sampling prior to treatment. Targeted proteomics of 327 proteins using MRM-MS was used to measure plasma levels at baseline (including pre-treatment and early treatment blood samples) and assess potential clinical association. RESULTS: Patients were categorized by duration of response: long-term responders [PFS ≥ 24 months (n = 7)], normal responders [3 < PFS < 24 months (n = 10)] and poor responders [PFS ≤ 3 months (n = 5)]. Several proteins were identified as differentially expressed between long-term responders and poor responders, including DPP4, KIT and LUM. Next, using machine learning algorithms, we evaluated the classification potential of 40 proteins. Finally, by integrating the different analytic methods, we selected 22 proteins as potential candidates for a blood-based prognostic signature of response to crizotinib in NSCLC patients harboring ALK fusion. CONCLUSION: In conjunction with ALK mutation, the expression of this proteomic signature may represent a liquid biopsy-based marker of long-term response to crizotinib in NSCLC. Expanding the utility of prognostic biomarkers of response duration could influence choice of therapy, therapeutic sequencing, and potentially the need for alternative or combination therapy.Trial registration ClinicalTrials.gov, NCT02041468. Registered 22 January 2014, https://clinicaltrials.gov/ct2/show/NCT02041468?term=NCT02041468&rank=1.

Learning to use past evidence in a sophisticated world model.

Humans and other animals are able to discover underlying statistical structure in their environments and exploit it to achieve efficient and effective performance. However, such structure is often difficult to learn and use because it is obscure, involving long-range temporal dependencies. Here, we analysed behavioural data from an extended experiment with rats, showing that the subjects learned the underlying statistical structure, albeit suffering at times from immediate inferential imperfections as to their current state within it. We accounted for their behaviour using a Hidden Markov Model, in which recent observations are integrated with evidence from the past. We found that over the course of training, subjects came to track their progress through the task more accurately, a change that our model largely attributed to improved integration of past evidence. This learning reflected the structure of the task, decreasing reliance on recent observations, which were potentially misleading.

The Web-Surf Task: A translational model of human decision-making.

Animal models of decision-making are some of the most highly regarded psychological process models; however, there remains a disconnection between how these models are used for pre-clinical applications and the resulting treatment outcomes. This may be due to untested assumptions that different species recruit the same neural or psychological mechanisms. We propose a novel human foraging paradigm (Web-Surf Task) that we translated from a rat foraging paradigm (Restaurant Row) to evaluate cross-species decision-making similarities. We examined behavioral parallels in human and non-human animals using the respective tasks. We also compared two variants of the human task, one using videos and the other using photos as rewards, by correlating revealed and stated preferences. We demonstrate similarities in choice behaviors and decision reaction times in human and rat subjects. Findings also indicate that videos yielded more reliable and valid results. The joint use of the Web-Surf Task and Restaurant Row is therefore a promising approach for functional translational research, aiming to bridge pre-clinical and clinical lines of research using analogous tasks.

Aging impairs deliberation and behavioral flexibility in inter-temporal choice.

Inter-temporal choice depends on multiple, interacting systems, some of which may be compromised with age. Some of these systems may be responsible for ongoing trial-by-trial choice strategies. Some may represent the consequences of action. Some may be necessary for the coupling between anticipated consequences and strategies currently in use, flexibly guiding behavior. When faced with a difficult decision, rats will orient back and forth, a behavior termed "vicarious trial and error" (VTE). Recent experiments have linked the occurrence of VTE to hippocampal search processes and behavioral flexibility. We tested 5 month (n = 6), 9 month (n = 8) and over-27 month-old (n = 10) rats on a Spatial Adjusting Delay Discounting task to examine how aging impacted lap-by-lap strategies and VTE during inter-temporal choice. Rats chose between spatially separated food goals that provided a smaller-sooner or larger-later reward. On each lap, the delay to the larger-later reward was adjusted as a function of the rat's decisions, increasing by 1 s after delayed-side choices and decreasing by 1 s after non-delayed side choices. The strategies that aged rats used differed from those used in young and adult rats. Moreover, aged rats produced reliably more VTE behaviors, for protracted periods of time, uncoupled from behavioral flexibility.

The effect of probability discounting on reward seeking: a three-dimensional perspective.

Rats will work for electrical stimulation of the medial forebrain bundle. The rewarding effect arises from the volleys of action potentials fired by the stimulation and subsequent spatio-temporal integration of their post-synpatic impact. The proportion of time allocated to self-stimulation depends on the intensity of the rewarding effect as well as on other key determinants of decision-making, such as subjective opportunity costs and reward probability. We have proposed that a 3D model relating time allocation to the intensity and cost of reward can distinguish manipulations acting prior to the output of the spatio-temporal integrator from those acting at or beyond it. Here, we test this proposition by varying reward probability, a variable that influences the computation of payoff in the 3D model downstream from the output of the integrator. On riskless trials, reward was delivered on every occasion that the rat held down the lever for a cumulative duration called the "price," whereas on risky trials, reward was delivered with probability 0.75 or 0.50. According to the model, the 3D structure relating time allocation to reward intensity and price is shifted leftward along the price axis by reductions in reward probability; the magnitude of the shift estimates the change in subjective probability. The predictions were borne out: reducing reward probability shifted the 3D structure systematically along the price axis while producing only small, inconsistent displacements along the pulse-frequency axis. The results confirm that the model can accurately distinguish manipulations acting at or beyond the spatio-temporal integrator and strengthen the conclusions of previous studies showing similar shifts following dopaminergic manipulations. Subjective and objective reward probabilities appeared indistinguishable over the range of 0.5 ≤ p ≤ 1.0.

Optimal indolence: a normative microscopic approach to work and leisure.

Dividing limited time between work and leisure when both have their attractions is a common everyday decision. We provide a normative control-theoretic treatment of this decision that bridges economic and psychological accounts. We show how our framework applies to free-operant behavioural experiments in which subjects are required to work (depressing a lever) for sufficient total time (called the price) to receive a reward. When the microscopic benefit-of-leisure increases nonlinearly with duration, the model generates behaviour that qualitatively matches various microfeatures of subjects' choices, including the distribution of leisure bout durations as a function of the pay-off. We relate our model to traditional accounts by deriving macroscopic, molar, quantities from microscopic choices.

Validation and extension of the reward-mountain model.

The reward-mountain model relates the vigor of reward seeking to the strength and cost of reward. Application of this model provides information about the stage of processing at which manipulations such as drug administration, lesions, deprivation states, and optogenetic interventions act to alter reward seeking. The model has been updated by incorporation of new information about frequency following in the directly stimulated neurons responsible for brain stimulation reward and about the function that maps objective opportunity costs into subjective ones. The behavioral methods for applying the model have been updated and improved as well. To assess the impact of these changes, two related predictions of the model that were supported by earlier work have been retested: (1) altering the duration of rewarding brain stimulation should change the pulse frequency required to produce a reward of half-maximal intensity, and (2) this manipulation should not change the opportunity cost at which half-maximal performance is directed at earning a maximally intense reward. Prediction 1 was supported in all six subjects, but prediction 2 was supported in only three. The latter finding is interpreted to reflect recruitment, at some stimulation sites, of a heterogeneous reward substrate comprising dual, parallel circuits that integrate the stimulation-induced neural signals.

At what stage of neural processing does cocaine act to boost pursuit of rewards?

Dopamine-containing neurons have been implicated in reward and decision making. One element of the supporting evidence is that cocaine, like other drugs that increase dopaminergic neurotransmission, powerfully potentiates reward seeking. We analyze this phenomenon from a novel perspective, introducing a new conceptual framework and new methodology for determining the stage(s) of neural processing at which drugs, lesions and physiological manipulations act to influence reward-seeking behavior. Cocaine strongly boosts the proclivity of rats to work for rewarding electrical brain stimulation. We show that the conventional conceptual framework and methods do not distinguish between three conflicting accounts of how the drug produces this effect: increased sensitivity of brain reward circuitry, increased gain, or decreased subjective reward costs. Sensitivity determines the stimulation strength required to produce a reward of a given intensity (a measure analogous to the KM of an enzyme) whereas gain determines the maximum intensity attainable (a measure analogous to the vmax of an enzyme-catalyzed reaction). To distinguish sensitivity changes from the other determinants, we measured and modeled reward seeking as a function of both stimulation strength and opportunity cost. The principal effect of cocaine was a two-fourfold increase in willingness to pay for the electrical reward, an effect consistent with increased gain or decreased subjective cost. This finding challenges the long-standing view that cocaine increases the sensitivity of brain reward circuitry. We discuss the implications of the results and the analytic approach for theories of how dopaminergic neurons and other diffuse modulatory brain systems contribute to reward pursuit, and we explore the implications of the conceptual framework for the study of natural rewards, drug reward, and mood.

Rattus Psychologicus: construction of preferences by self-stimulating rats.

Behavioral economists have proposed that human preferences are constructed during their elicitation and are thus influenced by the elicitation procedure. For example, different preferences are expressed when options are encountered one at a time or concurrently. This phenomenon has been attributed to differences in the "evaluability" of a particular attribute when comparison to an option with a different value of this attribute is or is not available. Research on the preferences of laboratory animals has often been carried out by means of operant-conditioning methods. Formal treatments of operant behavior relate preferences to variables such as the strength and cost of reward but do not address the evaluability of these variables. Two experiments assessed the impact of procedural factors likely to alter the evaluability of an opportunity cost ("price"): the work time required for a rat to earn a train of rewarding electrical brain stimulation. The results support the notion that comparison between recently encountered prices is necessary to render the price variable highly evaluable. When price is held constant over many trials and test sessions, the evaluability of this variable appears to decline. Implications are discussed for the design of procedures for estimating subjective reward strengths and costs in operant-conditioning experiments aimed at characterizing, identifying and understanding neural circuitry underlying evaluation and choice.