Inter-individual variations in circadian misalignment-induced NAFLD pathophysiology in mice.

Pathological consequences of circadian misalignment, such as shift work, show considerable individual differences, but the lack of mechanistic understanding hinders precision prevention to prevent and mitigate disease symptoms. Here, we employed an integrative approach involving physiological, transcriptional, and histological phenotypes to examine inter-individual differences in pre-symptomatic pathological progression, preceding irreversible disease onset, in wild-type mice exposed to chronic jet-lag (CJL). We observed that CJL markedly increased the prevalence of hepatic steatosis with pronounced inter-individual differences. Stratification of individual mice based on CJL-induced hepatic transcriptomic signature, validated by histopathological analysis, pinpoints dysregulation of lipid metabolism. Moreover, the period and power of intrinsic behavioral rhythms were found to significantly correlate with CJL-induced gene signatures. Together, our results suggest circadian rhythm robustness of the animals contributes to inter-individual variations in pathogenesis of circadian misalignment-induced diseases and raise the possibility that these physiological indicators may be available for predictive hallmarks of circadian rhythm disorders.

Micronized Cellular Adipose Matrix Promotes the Therapeutic Effect of an Artificial Nerve Conduit in Peripheral Nerve Gap Injury.

BACKGROUND: Stromal vascular fraction (SVF) isolated from adipose tissue has been shown to be beneficial for treating peripheral nerve injuries. Micronized cellular adipose matrix (MCAM) is an SVF-rich micronized fat tissue obtained by a series of simple mechanical processes. This study assessed the therapeutic effect of MCAM for peripheral nerve injury. METHODS: Microscopic evaluation of the cell phenotype and functions was performed to determine the adipose-derived stem cell content of the MCAM. An artificial nerve conduit (ANC) filled with MCAM was implanted into a sciatic nerve defect in immunodeficient mice. Comparisons of this treatment with an autograft, an ANC filled with SVF cells, and an ANC alone were made based on electrophysiologic characteristics, Sciatic Functional Index, and histologic analyses of regenerated nerve fiber and myelination using electron microscopy, and the preventive effect on innervated muscle atrophy. RESULTS: MCAM contained many cells with a phenotype and differentiation potency similar to those of ADSCs. The implantation experiment indicated that MCAM enhanced the efficiency of functional and structural recovery and prevented atrophy of the innervated muscle. These effects were significantly improved compared with the control group (ANC only) and comparable to those in the SVF group, whereas the improvement did not reach the same level of the autograft group. CONCLUSION: Injection of MCAM into an ANC accelerated nerve regeneration compared with use of an ANC alone, which indicates that MCAM is a promising transplant material for treatment of peripheral nerve injury and an alternative to use of SVF cells. CLINICAL RELEVANCE STATEMENT: Micronized cellular adipose matrix, which can be harvested and isolated from adipose tissue with a simple device, has been shown for the first time to be highly useful as an implantable material for new peripheral nerve regeneration.

Association of social jetlag and eating patterns with sleep quality and daytime sleepiness in Japanese high school students.

A high prevalence of excessive daytime sleepiness and poor sleep quality has been reported in adolescents, but the effects of social jetlag on sleep quality and daytime sleepiness are unclear. Therefore, we assessed the association of sleep and eating patterns with daytime sleepiness and sleep quality among a total of 756 Japanese high school students. Participants completed the Pittsburgh Sleep Quality Index to evaluate sleep quality, the Pediatric Daytime Sleepiness Scale to evaluate daytime sleepiness, and an 8-day sleep diary. Data on average sleep duration, social jetlag, midsleep on free days sleep corrected, and the differences in the first and last meal timing between school days and non-school days were obtained from participants' sleep diaries. The results reveal that social jetlag is associated with differences in the first meal timing between school days and non-school days, and that social jetlag of more than 2 hr is associated with extremely poor sleep quality and excessive daytime sleepiness in Japanese high school students. Our findings suggest that reducing social jetlag to within a 2-hr window is important to prevent poor sleep quality and excessive daytime sleepiness for this population.

Cobalt nanoparticle supported on layered double hydroxide: Effect of nanoparticle size on catalytic hydrogen production by NaBH4 hydrolysis.

Catalytic hydrolysis of sodium borohydride (NaBH4) is a promising method to provide clean hydrogen (H2) energy for portable devices. Therefore, designing a non-noble metal catalyst that performs well in this hydrolysis is essential. Cobalt-nanoparticles (Co-NPs) supported on magnesium-aluminium layered double hydroxide (LDH) with various mean diameter were synthesized by changing concentration of cobalt-citrate anion (Co-citrate) precursor used for ion exchange with the LDH host. Then the Co-citrate intercalated LDHs were reduced with NaBH4 to form Co-NPs. Evidence of successful intercalation was shown by X-ray diffraction (XRD) and Fourier-Transform Infrared spectroscopy (FT-IR). Transmission Electron Microscope (TEM) and Scanning TEM (STEM) observations revealed that Co-NPs were in metallic state and their mean diameter increased with the concentration of Co-citrate solution. Nitrogen physisorption isotherms showed that the surface structure of LDHs transformed from non-porous to mesoporous after chemical reduction, which indicated that the Co-NPs were formed in the interlayer of LDHs. Catalytic hydrolysis of NaBH4 at 25 °C clarified that the catalyst synthesized from 6 mM Co-citrate solution showed the highest H2 generation rate of 4520 ± 251 mL min-1·gCo-1, indicating the catalyst had the optimum size of Co-NP. This activity could be considered relatively higher compared to unsupported cobalt and many other supported cobalt-base catalysts previously reported. It was also clearly shown that size of Co-NPs supported on LDH could be a significant parameter as it allowed better accessibility of reactants to the active catalyst surface to obtain maximum activity. For this optimum catalyst, the activation energy was evaluated to be 56.9 kJ mol-1. Although the catalyst was able to achieve almost the same conversion when the catalyst was repeatedly tested five times under the same condition, the catalytic activity decreased gradually. Overall, it could be revealed that Co-NPs supported on LDHs have a huge potential to be used for H2 energy production.

Quantitative morphometric analysis of molar teeth and alveolar bone using micro-computed tomography in aged mice.

OBJECTIVES: Irreversible morphological regressions of the teeth or related structures in older people can diminish their overall health. However, research on human aging in dentistry is complicated by several confounding factors. In this study, we conducted a morphometric analysis of the mandibular second molars and surrounding alveolar bone in C57BL/6 mice to evaluate age-related changes in the oral cavity. METHODS: The animals were divided into five groups based on their age: 4 weeks (juvenile mice; n = 5); 20 weeks (n = 7), 50 weeks (n = 5), 77 weeks (n = 7), and 100 weeks (n = 5); changes were evaluated using micro-computed tomography. RESULTS: The molars of juvenile mice had sharp and pointed cusps and presented maximum heights. With age and occlusal wear, the cusp heights demonstrated a significant decrease (up to 75%) until the last stage of life. Conversely, apparent lesions were not observed on the basal portion of the crown, even in the most heavily worn teeth. The roots of the molars continued to grow in length at 4 weeks of age. Alveolar bone resorption begins to occur in middle age and continues throughout life. The proportion of vertical bone loss reached approximately 40% of the entire root length, demonstrating a remarkable increase between weeks 77 and 100. CONCLUSIONS: Overall, these morphological changes were similar to those observed in humans. Therefore, it might be appropriate to use aged mice as an experimental model for basic and clinical research in geriatric dentistry.

Tonically Active Neurons in the Monkey Dorsal Striatum Signal Outcome Feedback during Trial-and-error Search Behavior.

An animal's choice behavior is shaped by the outcome feedback from selected actions in a trial-and-error approach. Tonically active neurons (TANs), presumed cholinergic interneurons in the striatum, are thought to be involved in the learning and performance of reward-directed behaviors, but it remains unclear how TANs are involved in shaping reward-directed choice behaviors based on the outcome feedback. To this end, we recorded activity of TANs from the dorsal striatum of two macaque monkeys (Macaca fuscata; 1 male, 1 female) while they performed a multi-step choice task to obtain multiple rewards. In this task, the monkeys first searched for a rewarding target from among three alternatives in a trial-and-error manner and then earned additional rewards by repeatedly choosing the rewarded target. We found that a considerable proportion of TANs selectively responded to either the reward or the no-reward outcome feedback during the trial-and-error search, but these feedback responses were not observed during repeat trials. Moreover, the feedback responses of TANs were similarly observed in any search trials, without distinctions regarding the predicted probability of rewards and the location of chosen targets. Unambiguously, TANs detected reward and no-reward feedback specifically when the monkeys performed trial-and-error searches, in which the monkeys were learning the value of the targets and adjusting their subsequent choice behavior based on the reward and no-reward feedback. These results suggest that striatal cholinergic interneurons signal outcome feedback specifically during search behavior, in circumstances where the choice outcomes cannot be predicted with certainty by the animals.

Topographic distinction in long-term value signals between presumed dopamine neurons and presumed striatal projection neurons in behaving monkeys.

Nigrostriatal dopamine (DA) projections are anatomically organized along the dorsolateral-ventromedial axis, conveying long-term value signals to the striatum for shaping actions toward multiple future rewards. The present study examines whether the topographic organization of long-term value signals are observed upon activity of presumed DA neurons and presumed striatal projection neurons (phasically active neurons, PANs), as predicted based on anatomical literature. Our results indicate that DA neurons in the dorsolateral midbrain encode long-term value signals on a short timescale, while ventromedial midbrain DA neurons encode such signals on a relatively longer timescale. Activity of the PANs in the dorsal striatum is more heterogeneous for encoding long-term values, although significant differences in long-term value signals were observed between the caudate nucleus and putamen. These findings suggest that topographic DA signals for long-term values are not simply transferred to striatal neurons, possibly due to the contribution of other projections to the striatum.

Chronic circadian misalignment accelerates immune senescence and abbreviates lifespan in mice.

Modern society characterized by a 24/7 lifestyle leads to misalignment between environmental cycles and endogenous circadian rhythms. Persisting circadian misalignment leads to deleterious effects on health and healthspan. However, the underlying mechanism remains not fully understood. Here, we subjected adult, wild-type mice to distinct chronic jet-lag paradigms, which showed that long-term circadian misalignment induced significant early mortality. Non-biased RNA sequencing analysis using liver and kidney showed marked activation of gene regulatory pathways associated with the immune system and immune disease in both organs. In accordance, we observed enhanced steatohepatitis with infiltration of inflammatory cells. The investigation of senescence-associated immune cell subsets from the spleens and mesenteric lymph nodes revealed an increase in PD-1+CD44high CD4 T cells as well as CD95+GL7+ germinal center B cells, indicating that the long-term circadian misalignment exacerbates immune senescence and consequent chronic inflammation. Our results underscore immune homeostasis as a pivotal interventional target against clock-related disorders.

Enhanced metastatic growth after local tumor resection in the presence of synchronous metastasis in a mouse allograft model of neuroblastoma.

PURPOSE: We investigated how local tumor resection affects metastatic lesions in neuroblastoma. METHODS: MYCN Tg tumor-derived cells were injected subcutaneously into 129+Ter/SvJcl wild-type mice. First, the frequency of metastasis-bearing mice was investigated immunohistochemically (metastatic ratio) at endpoint or post-injection day (PID) 90. Second, the threshold volume of local tumor in mice bearing microscopic lymph node metastasis (mLNM) was investigated at PID 30. Finally, local tumors were resected after exceeding the threshold. Mice were divided into local tumor resection (Resection) and observation (Observation) groups, and the metastatic ratio and volume of LNM were compared between the groups at endpoint or PID 74. RESULTS: The metastatic ratio without local resection was 88% at PID 78-90. The threshold local tumor volume in the mice with mLNM was 745 mm3 at PID 30, so local tumors were resected after exceeding 700 mm3. The metastatic ratio and LNM volume were significantly greater in the Resection group (n = 16) than in the Observation group (n = 16) (94% vs. 38%, p < 0.001; 2092 ± 2310 vs. 275 ± 218 mm3, p < 0.01; respectively) at PID 50-74. CONCLUSION: Local tumor resection might augment the growth of synchronous microscopic metastases. Our results provide insights into the appropriate timing of local resection for high-risk neuroblastoma.

REV-ERBα and REV-ERBß function as key factors regulating Mammalian Circadian Output.

The circadian clock regulates behavioural and physiological processes in a 24-h cycle. The nuclear receptors REV-ERBα and REV-ERBß are involved in the cell-autonomous circadian transcriptional/translational feedback loops as transcriptional repressors. A number of studies have also demonstrated a pivotal role of REV-ERBs in regulation of metabolic, neuronal, and inflammatory functions including bile acid metabolism, lipid metabolism, and production of inflammatory cytokines. Given the multifunctional role of REV-ERBs, it is important to elucidate the mechanism through which REV-ERBs exert their functions. To this end, we established a Rev-erbα/Rev-erbß double-knockout mouse embryonic stem (ES) cell model and analyzed the circadian clock and clock-controlled output gene expressions. A comprehensive mRNA-seq analysis revealed that the double knockout of both Rev-erbα and Rev-erbß does not abrogate expression rhythms of E-box-regulated core clock genes but drastically changes a diverse set of other rhythmically-expressed output genes. Of note, REV-ERBα/ß deficiency does not compromise circadian expression rhythms of PER2, while REV-ERB target genes, Bmal1 and Npas2, are significantly upregulated. This study highlight the relevance of REV-ERBs as pivotal output mediators of the mammalian circadian clock.

Incremental Growth Lines in Mouse Molar Dentin Represent 8-hr Ultradian Rhythm.

Rhythmic incremental growth lines occur in dental hard tissues of vertebrates, and dentinogenesis in rodent incisors is suggested to be controlled by the 24-hr circadian clock. Rodent incisors continue to grow throughout the animal's life; however, similar to human teeth, rodent molars stop growing after crown formation. This similarity suggests that the mouse molar is an excellent model to understand the molecular mechanisms underlying growth of human teeth. However, not much is known about the rhythmic dentinogenesis in mouse molars. Here, we investigated the incremental growth lines in mouse molar dentin using tetracycline as the growth marker. The incremental growth lines were observed to be generated at approximately 8-hr intervals in wild-type mice housed under 12:12 hr light-dark conditions. Moreover, the 8-hr rhythmic increments persisted in the wild-type and Bmal1-/- mice housed in constant darkness, where Bmal1-/- mice become behaviorally arrhythmic. These results revealed that the dentinogenesis in mouse molars underlie the ultradian rhythms with around 8-hr periodicity. Further, the circadian clock does not seem to be involved in this process, providing new insight into the mechanisms involved in the tooth growth.

Robust circadian clock oscillation and osmotic rhythms in inner medulla reflecting cortico-medullary osmotic gradient rhythm in rodent kidney.

Circadian clocks in mammals function in most organs and tissues throughout the body. Various renal functions such as the glomerular filtration and excretion of electrolytes exhibit circadian rhythms. Although it has been reported that the expression of the clock genes composing molecular oscillators show apparent daily rhythms in rodent kidneys, functional variations of regional clocks are not yet fully understood. In this study, using macroscopic bioluminescence imaging method of the PER2::Luciferase knock-in mouse kidney, we reveal that strong and robust circadian clock oscillation is observed in the medulla. In addition, the osmotic pressure in the inner medulla shows apparent daily fluctuation, but not in the cortex. Quantitative-PCR analysis of the genes contributing to the generation of high osmotic pressure or the water re-absorption in the inner medulla, such as vasopressin receptors (V1aR, V2R), urea transporter (UT-A2) and water channel (Aqp2) show diurnal variations as well as clock genes. Deficiency of an essential clock gene Bmal1 impairs day-night variations of osmotic pressure gradient in the inner medulla, suggesting that circadian clocks in the medulla part of the kidney may regulate the circadian rhythm of cortico-medullary osmotic pressure gradient, and may contribute physiological day-night rhythm of urination.

Distinct Functions of the Primate Putamen Direct and Indirect Pathways in Adaptive Outcome-Based Action Selection.

Cortico-basal ganglia circuits are critical regulators of reward-based decision making. Reinforcement learning models posit that action reward value is encoded by the firing activity of striatal medium spiny neurons (MSNs) and updated upon changing reinforcement contingencies by dopamine (DA) signaling to these neurons. However, it remains unclear how the anatomically distinct direct and indirect pathways through the basal ganglia are involved in updating action reward value under changing contingencies. MSNs of the direct pathway predominantly express DA D1 receptors and those of the indirect pathway predominantly D2 receptors, so we tested for distinct functions in behavioral adaptation by injecting D1 and D2 receptor antagonists into the putamen of two macaque monkeys performing a free choice task for probabilistic reward. In this task, monkeys turned a handle toward either a left or right target depending on an asymmetrically assigned probability of large reward. Reward probabilities of left and right targets changed after 30-150 trials, so the monkeys were required to learn the higher-value target choice based on action-outcome history. In the control condition, the monkeys showed stable selection of the higher-value target (that more likely to yield large reward) and kept choosing the higher-value target regardless of less frequent small reward outcomes. The monkeys also made flexible changes of selection away from the high-value target when two or three small reward outcomes occurred randomly in succession. DA D1 antagonist injection significantly increased the probability of the monkey switching to the alternate target in response to successive small reward outcomes. Conversely, D2 antagonist injection significantly decreased the switching probability. These results suggest distinct functions of D1 and D2 receptor-mediated signaling processes in action selection based on action-outcome history, with D1 receptor-mediated signaling promoting the stable choice of higher-value targets and D2 receptor-mediated signaling promoting a switch in action away from small reward outcomes. Therefore, direct and indirect pathways appear to have complementary functions in maintaining optimal goal-directed action selection and updating action value, which are dependent on D1 and D2 DA receptor signaling.

Characteristics of fast-spiking neurons in the striatum of behaving monkeys.

Inhibitory interneurons are the fundamental constituents of neural circuits that organize network outputs. The striatum as part of the basal ganglia is involved in reward-directed behaviors. However, the role of the inhibitory interneurons in this process remains unclear, especially in behaving monkeys. We recorded the striatal single neuron activity while monkeys performed reward-directed hand or eye movements. Presumed parvalbumin-containing GABAergic interneurons (fast-spiking neurons, FSNs) were identified based on narrow spike shapes in three independent experiments, though they were a small population (4.2%, 42/997). We found that FSNs are characterized by high-frequency and less-bursty discharges, which are distinct from the basic firing properties of the presumed projection neurons (phasically active neurons, PANs). Besides, the encoded information regarding actions and outcomes was similar between FSNs and PANs in terms of proportion of neurons, but the discharge selectivity was higher in PANs than that of FSNs. The coding of actions and outcomes in FSNs and PANs was consistently observed under various behavioral contexts in distinct parts of the striatum (caudate nucleus, putamen, and anterior striatum). Our results suggest that FSNs may enhance the discharge selectivity of postsynaptic output neurons (PANs) in encoding crucial variables for a reward-directed behavior.

Effect of Multiple Clock Gene Ablations on the Circadian Period Length and Temperature Compensation in Mammalian Cells.

Most organisms have cell-autonomous circadian clocks to coordinate their activity and physiology according to 24-h environmental changes. Despite recent progress in circadian studies, it is not fully understood how the period length and the robustness of mammalian circadian rhythms are determined. In this study, we established a series of mouse embryonic stem cell (ESC) lines with single or multiplex clock gene ablations using the CRISPR/Cas9-based genome editing method. ESC-based in vitro circadian clock formation assay shows that the CRISPR-mediated clock gene disruption not only reproduces the intrinsic circadian molecular rhythms of previously reported mice tissues and cells lacking clock genes but also reveals that complexed mutations, such as CKIδ(m/m):CKIε(+/m):Cry2(m/m) mutants, exhibit an additively lengthened circadian period. By using these mutant cells, we also investigated the relation between period length alteration and temperature compensation. Although CKIδ-deficient cells slightly affected the temperature insensitivity of period length, we demonstrated that the temperature compensation property is largely maintained in all mutants. These results show that the ESC-based assay system could offer a more systematic and comprehensive approach to the genotype-chronotype analysis of the intracellular circadian clockwork in mammals.

Parathyroid hormone resets the cartilage circadian clock of the organ-cultured murine femur.

BACKGROUND AND PURPOSE: The circadian clock governs endogenous day-night variations. In bone, the metabolism and growth show diurnal rhythms. The circadian clock is based on a transcription-translation feedback loop composed of clock genes including Period2 (Per2), which encodes the protein period circadian protein homolog 2. Because plasma parathyroid hormone (PTH) levels show diurnal variation, we hypothesized that PTH could carry the time information to bone and cartilage. In this study, we analyzed the effect of PTH on the circadian clock of the femur. PATIENTS AND METHODS: Per2::Luciferase (Per2::Luc) knock-in mice were used and their femurs were organ-cultured. The bioluminescence was measured using photomultiplier tube-based real-time bioluminescence monitoring equipment or real-time bioluminescence microscopic imaging devices. PTH or its vehicle was administered and the phase shifts were calculated. Immunohistochemistry was performed to detect PTH type 1 receptor (PTH1R) expression. RESULTS: Real-time bioluminescence monitoring revealed that PTH reset the circadian rhythm of the Per2::Luc activity in the femurs in an administration time-dependent and dose-dependent manner. Microscopic bioluminescence imaging revealed that Per2::Luc activity in the growth plate and the articular cartilage showed that the circadian rhythms and their phase shifts were induced by PTH. PTH1R was expressed in the growth plate cartilage. INTERPRETATION: In clinical practice, teriparatide (PTH (1-34)) treatment is widely used for osteoporosis. We found that PTH administration regulated the femoral circadian clock oscillation, particularly in the cartilage. Regulation of the local circadian clock by PTH may lead to a more effective treatment for not only osteoporosis but also endochondral ossification in bone growth and fracture repair.

Prolonged bioluminescence monitoring in mouse ex vivo bone culture revealed persistent circadian rhythms in articular cartilages and growth plates.

The bone is a metabolically active organ which undergoes repeated remodeling cycles of bone resorption and formation. In this study, we revealed a robust and extremely long-lasting circadian rhythm in ex vivo culture maintained for over six months from the femoral bone of a PERIOD2(Luciferase) mouse. Furthermore, we also identified robust circadian clocks in flat bones. High- or low-magnification real-time bioluminescence microscopic imaging revealed that the robust circadian rhythms emanated from the articular cartilage and the epiphyseal cartilage within the growth plate of juvenile animals. Stimulation by forskolin or dexamethasone treatment caused type 0 phase resetting, indicating canonical entraining properties of the bone clock. Together, our findings from long-term ex vivo culture revealed that "tissue-autonomous" circadian rhythm in the articular cartilage and the growth plate of femoral bone functions for several months even in an organ culture condition, and provided a useful in vitro assay system investigating the role of the biological clock in bone formation or development.

An in vitro ES cell-based clock recapitulation assay model identifies CK2α as an endogenous clock regulator.

We previously reported emergence and disappearance of circadian molecular oscillations during differentiation of mouse embryonic stem (ES) cells and reprogramming of differentiated cells, respectively. Here we present a robust and stringent in vitro circadian clock formation assay that recapitulates in vivo circadian phenotypes. This assay system first confirmed that a mutant ES cell line lacking Casein Kinase I delta (CKIδ) induced ∼3 hours longer period-length of circadian rhythm than the wild type, which was compatible with recently reported results using CKIδ null mice. In addition, this assay system also revealed that a Casein Kinase 2 alpha subunit (CK2α) homozygous mutant ES cell line developed significantly longer (about 2.5 hours) periods of circadian clock oscillations after in vitro or in vivo differentiation. Moreover, revertant ES cell lines in which mutagenic vector sequences were deleted showed nearly wild type periods after differentiation, indicating that the abnormal circadian period of the mutant ES cell line originated from the mutation in the CK2α gene. Since CK2α deficient mice are embryonic lethal, this in vitro assay system represents the genetic evidence showing an essential role of CK2α in the mammalian circadian clock. This assay was successfully applied for the phenotype analysis of homozygous mutant ES cells, demonstrating that an ES cell-based in vitro assay is available for circadian genetic screening.

Coding of the long-term value of multiple future rewards in the primate striatum.

Decisions maximizing benefits involve a tradeoff between the quantity of a reward and the cost of elapsed time until an animal receives it. The estimation of long-term reward values is critical to attain the most desirable outcomes over a certain period of time. Reinforcement learning theories have established algorithms to estimate the long-term reward values of multiple future rewards in which the values of future rewards are discounted as a function of how many steps of choices are necessary to achieve them. Here, we report that presumed striatal projection neurons represent the long-term values of multiple future rewards estimated by a standard reinforcement learning model while monkeys are engaged in a series of trial-and-error choices and adaptive decisions for multiple rewards. We found that the magnitude of activity of a subset of neurons was positively correlated with the long-term reward values, and that of another subset of neurons was negatively correlated throughout the entire decision-making process in individual trials: from the start of the task trial, estimation of the values and their comparison among alternatives, choice execution, and evaluation of the received rewards. An idiosyncratic finding was that neurons showing negative correlations represented reward values in the near future (high discounting), while neurons showing positive correlations represented reward values not only in the near future, but also in the far future (low discounting). These findings provide a new insight that long-term value signals are embedded in two subsets of striatal neurons as high and low discounting of multiple future rewards.

Dopamine neurons learn to encode the long-term value of multiple future rewards.

Midbrain dopamine neurons signal reward value, their prediction error, and the salience of events. If they play a critical role in achieving specific distant goals, long-term future rewards should also be encoded as suggested in reinforcement learning theories. Here, we address this experimentally untested issue. We recorded 185 dopamine neurons in three monkeys that performed a multistep choice task in which they explored a reward target among alternatives and then exploited that knowledge to receive one or two additional rewards by choosing the same target in a set of subsequent trials. An analysis of anticipatory licking for reward water indicated that the monkeys did not anticipate an immediately expected reward in individual trials; rather, they anticipated the sum of immediate and multiple future rewards. In accordance with this behavioral observation, the dopamine responses to the start cues and reinforcer beeps reflected the expected values of the multiple future rewards and their errors, respectively. More specifically, when monkeys learned the multistep choice task over the course of several weeks, the responses of dopamine neurons encoded the sum of the immediate and expected multiple future rewards. The dopamine responses were quantitatively predicted by theoretical descriptions of the value function with time discounting in reinforcement learning. These findings demonstrate that dopamine neurons learn to encode the long-term value of multiple future rewards with distant rewards discounted.