Intraindividual Fluctuation in Optimism Under Daily Life Circumstances: A Longitudinal Study.

Optimism is typically conceptualized as a relatively static tendency regarding positive expectations about one's future. However, recent studies suggest that optimism may meaningfully fluctuate within individuals over time. To date, little is known about the characteristics of such state optimism and potential cultural difference in state optimism. Accordingly, we developed a Japanese version of the State Optimism Measure (J-SOM) and examined its validity and the nature of intraindividual state optimism fluctuations; we also examined relationships between the J-SOM and other measures of mental health, including trait optimism. We conducted two online longitudinal surveys with different time intervals (weekly, n = 97; monthly, n = 99) targeting university students. Results were largely consistent between the two surveys. We confirmed high factor validity and internal consistency of the J-SOM. The J-SOM showed significant correlations in expected directions with other measures such as depressive mood and subjective happiness. In addition, intraindividual changes in the J-SOM were associated with changes in mood and quality of daily life. Importantly, these associations between intraindividual change in optimism and in other variables were minimal for trait optimism. We also found that state optimism, compared with trait optimism, tended to show larger intraindividual changes over 1, 2, 3, 4, and 8 weeks. In summary, this study developed a translated version of the SOM and validated it, and then showed, for the first time, that state optimism can fluctuate within individuals in daily life over a span of several weeks. Supplementary Information: The online version contains supplementary material available at 10.1007/s42761-023-00224-y.

Opponent learning with different representations in the cortico-basal ganglia pathways can develop obsession-compulsion cycle.

Obsessive-compulsive disorder (OCD) has been suggested to be associated with impairment of model-based behavioral control. Meanwhile, recent work suggested shorter memory trace for negative than positive prediction errors (PEs) in OCD. We explored relations between these two suggestions through computational modeling. Based on the properties of cortico-basal ganglia pathways, we modeled human as an agent having a combination of successor representation (SR)-based system that enables model-based-like control and individual representation (IR)-based system that only hosts model-free control, with the two systems potentially learning from positive and negative PEs in different rates. We simulated the agent's behavior in the environmental model used in the recent work that describes potential development of obsession-compulsion cycle. We found that the dual-system agent could develop enhanced obsession-compulsion cycle, similarly to the agent having memory trace imbalance in the recent work, if the SR- and IR-based systems learned mainly from positive and negative PEs, respectively. We then simulated the behavior of such an opponent SR+IR agent in the two-stage decision task, in comparison with the agent having only SR-based control. Fitting of the agents' behavior by the model weighing model-based and model-free control developed in the original two-stage task study resulted in smaller weights of model-based control for the opponent SR+IR agent than for the SR-only agent. These results reconcile the previous suggestions about OCD, i.e., impaired model-based control and memory trace imbalance, raising a novel possibility that opponent learning in model(SR)-based and model-free controllers underlies obsession-compulsion. Our model cannot explain the behavior of OCD patients in punishment, rather than reward, contexts, but it could be resolved if opponent SR+IR learning operates also in the recently revealed non-canonical cortico-basal ganglia-dopamine circuit for threat/aversiveness, rather than reward, reinforcement learning, and the aversive SR + appetitive IR agent could actually develop obsession-compulsion if the environment is modeled differently.

Computational models of behavioral addictions: State of the art and future directions.

Non-pharmacological behavioral addictions, such as pathological gambling, videogaming, social networking, or internet use, are becoming major public health concerns. It is not yet clear how behavioral addictions could share many major neurobiological and behavioral characteristics with substance use disorders, despite the absence of direct pharmacological influences. A deeper understanding of the neurocognitive mechanisms of addictive behavior is needed, and computational modeling could be one promising approach to explain intricately entwined cognitive and neural dynamics. This review describes computational models of addiction based on reinforcement learning algorithms, Bayesian inference, and biophysical neural simulations. We discuss whether computational frameworks originally conceived to explain maladaptive behavior in substance use disorders can be effectively extended to non-substance-related behavioral addictions. Moreover, we introduce recent studies on behavioral addictions that exemplify the possibility of such extension and propose future directions.

Opponent Learning with Different Representations in the Cortico-Basal Ganglia Circuits.

The direct and indirect pathways of the basal ganglia (BG) have been suggested to learn mainly from positive and negative feedbacks, respectively. Since these pathways unevenly receive inputs from different cortical neuron types and/or regions, they may preferentially use different state/action representations. We explored whether such a combined use of different representations, coupled with different learning rates from positive and negative reward prediction errors (RPEs), has computational benefits. We modeled animal as an agent equipped with two learning systems, each of which adopted individual representation (IR) or successor representation (SR) of states. With varying the combination of IR or SR and also the learning rates from positive and negative RPEs in each system, we examined how the agent performed in a dynamic reward navigation task. We found that combination of SR-based system learning mainly from positive RPEs and IR-based system learning mainly from negative RPEs could achieve a good performance in the task, as compared with other combinations. In such a combination of appetitive SR-based and aversive IR-based systems, both systems show activities of comparable magnitudes with opposite signs, consistent with the suggested profiles of the two BG pathways. Moreover, the architecture of such a combination provides a novel coherent explanation for the functional significance and underlying mechanism of diverse findings about the cortico-BG circuits. These results suggest that particularly combining different representations with appetitive and aversive learning could be an effective learning strategy in certain dynamic environments, and it might actually be implemented in the cortico-BG circuits.

Corticocortical innervation subtypes of layer 5 intratelencephalic cells in the murine secondary motor cortex.

Feedback projections from the secondary motor cortex (M2) to the primary motor and sensory cortices are essential for behavior selection and sensory perception. Intratelencephalic (IT) cells in layer 5 (L5) contribute feedback projections to diverse cortical areas. Here we show that L5 IT cells participating in feedback connections to layer 1 (L1) exhibit distinct projection patterns, genetic profiles, and electrophysiological properties relative to other L5 IT cells. An analysis of the MouseLight database found that L5 IT cells preferentially targeting L1 project broadly to more cortical regions, including the perirhinal and auditory cortices, and innervate a larger volume of striatum than the other L5 IT cells. We found experimentally that in upper L5 (L5a), ER81 (ETV1) was found more often in L1-preferring IT cells, and in IT cells projecting to perirhinal/auditory regions than those projecting to primary motor or somatosensory regions. The perirhinal region-projecting L5a IT cells were synaptically connected to each other and displayed lower input resistance than contra-M2 projecting IT cells including L1-preferring and nonpreferring cells. Our findings suggest that M2-L5a IT L1-preferring cells exhibit stronger ER81 expression and broader cortical/striatal projection fields than do cells that do not preferentially target L1.

Dopamine ramps for accurate value learning under uncertainty.

Dopamine signals ramping towards reward timings have become widely reported, but their functions remain elusive. Through modeling analyses and experiments in mice, a recent study by Mikhael, Kim et al. shows that such signals represent reward prediction errors used for accurate value learning in conditions with uncertainty about upcoming state and its resolution by sensory feedback.

A Reinforcement Learning Approach to Understanding Procrastination: Does Inaccurate Value Approximation Cause Irrational Postponing of a Task?

Procrastination is the voluntary but irrational postponing of a task despite being aware that the delay can lead to worse consequences. It has been extensively studied in psychological field, from contributing factors, to theoretical models. From value-based decision making and reinforcement learning (RL) perspective, procrastination has been suggested to be caused by non-optimal choice resulting from cognitive limitations. Exactly what sort of cognitive limitations are involved, however, remains elusive. In the current study, we examined if a particular type of cognitive limitation, namely, inaccurate valuation resulting from inadequate state representation, would cause procrastination. Recent work has suggested that humans may adopt a particular type of state representation called the successor representation (SR) and that humans can learn to represent states by relatively low-dimensional features. Combining these suggestions, we assumed a dimension-reduced version of SR. We modeled a series of behaviors of a "student" doing assignments during the school term, when putting off doing the assignments (i.e., procrastination) is not allowed, and during the vacation, when whether to procrastinate or not can be freely chosen. We assumed that the "student" had acquired a rigid reduced SR of each state, corresponding to each step in completing an assignment, under the policy without procrastination. The "student" learned the approximated value of each state which was computed as a linear function of features of the states in the rigid reduced SR, through temporal-difference (TD) learning. During the vacation, the "student" made decisions at each time-step whether to procrastinate based on these approximated values. Simulation results showed that the reduced SR-based RL model generated procrastination behavior, which worsened across episodes. According to the values approximated by the "student," to procrastinate was the better choice, whereas not to procrastinate was mostly better according to the true values. Thus, the current model generated procrastination behavior caused by inaccurate value approximation, which resulted from the adoption of the reduced SR as state representation. These findings indicate that the reduced SR, or more generally, the dimension reduction in state representation, can be a potential form of cognitive limitation that leads to procrastination.

Rigid reduced successor representation as a potential mechanism for addiction.

Difficulty in cessation of drinking, smoking, or gambling has been widely recognized. Conventional theories proposed relative dominance of habitual over goal-directed control, but human studies have not convincingly supported them. Referring to the recently suggested "successor representation (SR)" of states that enables partially goal-directed control, we propose a dopamine-related mechanism that makes resistance to habitual reward-obtaining particularly difficult. We considered that long-standing behavior towards a certain reward without resisting temptation can (but not always) lead to a formation of rigid dimension-reduced SR based on the goal state, which cannot be updated. Then, in our model assuming such rigid reduced SR, whereas no reward prediction error (RPE) is generated at the goal while no resistance is made, a sustained large positive RPE is generated upon goal reaching once the person starts resisting temptation. Such sustained RPE is somewhat similar to the hypothesized sustained fictitious RPE caused by drug-induced dopamine. In contrast, if rigid reduced SR is not formed and states are represented individually as in simple reinforcement learning models, no sustained RPE is generated at the goal. Formation of rigid reduced SR also attenuates the resistance-dependent decrease in the value of the cue for behavior, makes subsequent introduction of punishment after the goal ineffective, and potentially enhances the propensity of nonresistance through the influence of RPEs via the spiral striatum-midbrain circuit. These results suggest that formation of rigid reduced SR makes cessation of habitual reward-obtaining particularly difficult and can thus be a mechanism for addiction, common to substance and nonsubstance reward.

Structure and Validity of Questionnaire for Oral Frail Screening.

Oral frailty is defined as the mild decline in oral function and located at the early and reversible stage of frailty. Therefore, early detection and early treatment of oral frailty is very useful. Simple and easy questionnaires, such as an oral frailty checklist, have been widely used for the screening and enlightenment of oral frailty of the Japanese people. We evaluate the structure and validity of the oral frailty checklist. The questionnaire of oral frailty was distributed for the citizens more than 50 years old from December 2018 to January 2019. The structural validity of the questionnaire is analyzed by structural equation modeling (SEM). The characteristics of the items are analyzed by Item Response Theory (IRT). The data of 725 subjects (360 men, 359 women, 6 no answer, mean age 71.3 ± 9.05) are analyzed. The questionnaire consisted of three latent variables. Items of "Brushing teeth at least twice a day", "Regular attendance of dental clinic", and "Using denture", had low discrimination ability. The questionnaire used in this study is a useful tool for the screening of oral frailty. However, its scoring system needs to be improved.

Does Last Year's Cost Predict the Present Cost? An Application of Machine Leaning for the Japanese Area-Basis Public Health Insurance Database.

The increasing healthcare cost imposes a large economic burden for the Japanese government. Predicting the healthcare cost may be a useful tool for policy making. A database of the area-basis public health insurance of one city was analyzed to predict the medical healthcare cost by the dental healthcare cost with a machine learning strategy. The 30,340 subjects who had continued registration of the area-basis public health insurance of Ebina city during April 2017 to September 2018 were analyzed. The sum of the healthcare cost was JPY 13,548,831,930. The per capita healthcare cost was JPY 446,567. The proportion of medical healthcare cost, medication cost, and dental healthcare cost was 78%, 15%, and 7%, respectively. By the results of the neural network model, the medical healthcare cost proportionally depended on the medical healthcare cost of the previous year. The dental healthcare cost of the previous year had a reducing effect on the medical healthcare cost. However, the effect was very small. Oral health may be a risk for chronic diseases. However, when evaluated by the healthcare cost, its effect was very small during the observation period.

Film-Thickness-Controllable System for Preparing Silver Nanofilms through Absorbance Monitoring of the Thickness during a Silver-Mirror Reaction.

An innovative technique is proposed for forming silver thin films of nanometer-order thickness via a silver-mirror reaction. This approach is made possible by the real-time monitoring of the thickness of a silver thin film formed on the edge surface of a fiber core during the silver-mirror reaction using a homemade absorbance measurement system. The monitored absorbance value increases as silver plating progresses, and the relationship between the absorbance values and the thickness of the silver thin film is linear in the thickness range from approximately 30 to 60 nm. This technique was applied to the preparation of a fiber-optic surface plasmon resonance (FO-SPR) sensor. The sensor was successfully used to measure sucrose solutions with concentrations of less than 16% (w/v). The sensitivity of the sensor probe was estimated to be 2205 nm/RIU in the refractive index range of 1.333 - 1.357. The relative standard deviation of the wavelength shift obtained from measurements using different sensor probes was estimated to be less than 3.3%.

[Dynamical Systems Approach: An Elementary Introduction and Application to Research on Dopamine and Reinforcement Learning].

In order to analyze the dynamic behavior of neural systems such as oscillations or rhythms, an approach based on the dynamical systems theory may be useful. In the first part of this article, we present an elementary introduction to that approach based on the materials for an author's lecture. In the second part, we introduce our own study related to dopamine and reinforcement learning using that approach, in which we assumed the decay of learned values and propose a possible mechanism of the effects of dopamine depletion on motivation.

Non-action Learning: Saving Action-Associated Cost Serves as a Covert Reward.

"To do or not to do" is a fundamental decision that has to be made in daily life. Behaviors related to multiple "to do" choice tasks have long been explained by reinforcement learning, and "to do or not to do" tasks such as the go/no-go task have also been recently discussed within the framework of reinforcement learning. In this learning framework, alternative actions and/or the non-action to take are determined by evaluating explicitly given (overt) reward and punishment. However, we assume that there are real life cases in which an action/non-action is repeated, even though there is no obvious reward or punishment, because implicitly given outcomes such as saving physical energy and regret (we refer to this as "covert reward") can affect the decision-making. In the current task, mice chose to pull a lever or not according to two tone cues assigned with different water reward probabilities (70% and 30% in condition 1, and 30% and 10% in condition 2). As the mice learned, the probability that they would choose to pull the lever decreased (<0.25) in trials with a 30% reward probability cue (30% cue) in condition 1, and in trials with a 10% cue in condition 2, but increased (>0.8) in trials with a 70% cue in condition 1 and a 30% cue in condition 2, even though a non-pull was followed by neither an overt reward nor avoidance of overt punishment in any trial. This behavioral tendency was not well explained by a combination of commonly used Q-learning models, which take only the action choice with an overt reward outcome into account. Instead, we found that the non-action preference of the mice was best explained by Q-learning models, which regarded the non-action as the other choice, and updated non-action values with a covert reward. We propose that "doing nothing" can be actively chosen as an alternative to "doing something," and that a covert reward could serve as a reinforcer of "doing nothing."

Nutritional Status and Oral Frailty: A Community Based Study.

Compromised oral health can alter food choices. Poor masticatory function leads to imbalanced food intake and undesirable nutritional status. The associations among nutritional status, oral health behavior, and self-assessed oral functions status were investigated using a community-based survey. In total, 701 subjects more than 50 years old living Ebina city located southwest of the capital Tokyo were investigated. The number of remaining teeth was counted by dental hygienists. Oral health behavior and self-assessed oral functions were evaluated by oral frailty checklist. Nutritional status was evaluated by the brief-type self-administered diet history questionnaire using Dietary Reference Intakes for Japanese as reference. More than 80% of subjects' intakes of vitamin B12, pantothenic acid, copper, and proteins were sufficient. In contrast, only 19% of subjects' intake of vitamin A was sufficient and 35.5% for vitamin B1. More than 90% of subjects' intakes of vitamin D and vitamin K were sufficient. Only 35.5% of subjects' intakes of dietary fiber were sufficient. Overall, 88.9% of subjects had excess salt. The number of remaining teeth was not correlated with nutritional intakes. Oral health behavior significantly correlated with nutritional intakes. Oral functions are important for food choice; however, oral functions were not directly correlated with nutritional intakes. Comprehensive health instructions including nutrition and oral health education is necessary for health promotion.

K

The two-particle momentum correlation function of a K^{-}p pair from high-energy nuclear collisions is evaluated in the K[over ¯]N-πΣ-πΛ coupled-channel framework. The effects of all coupled channels together with the Coulomb potential and the threshold energy difference between K^{-}p and K[over ¯]^{0}n are treated completely for the first time. Realistic potentials based on the chiral SU(3) dynamics are used which fit the available scattering data. The recently measured correlation function is found to be well reproduced by allowing variations of the source size and the relative weight of the source function of πΣ with respect to that of K[over ¯]N. The predicted K^{-}p correlation function from larger systems indicates that the investigation of its source size dependence is useful in providing further constraints in the study of the K[over ¯]N interaction.

Caution in exploring the effects of distant past outcomes on sequential choices.

We sometimes make sequential decisions depending solely on the immediate past outcomes, e.g., according to the Win-Stay-Lose-Switch rule. In other occasions, we make decisions depending also on the distant past outcomes. It appears of interest to distinguish these two cases based on the generated choice sequences. At first glance, it may seem straightforward to distinguish the two cases by examining whether the rate of reselecting the same option that was chosen in the distant past, for example, at two trials before, depends on the outcome obtained there. However, such naive analysis can theoretically lead to detection of spurious dependence of three different types. Whereas two of them can easily be avoided by calculating the rate of reselection separately for each case sorted by the choice and outcome at the previous trial, the third type of spurious dependence appears after being sorted by the previous choice and outcome. Here we show how such spurious dependence appears. This article exemplifies the need for caution in analyzing the limited number of sequential choices.

Development of an on-chip sample injection system with a 6-port valve incorporated in a microchip.

Micro-flow-injection analysis (µFIA) is amenable to high-throughput systems with lower consumption of sample and reagent volumes. On-chip sample injection methods are important to prevent reduced analytical performance associated with dead volumes and diffusion of sample solutions. In this study, we have developed an on-chip sample injection system with a small-sized 6-port valve incorporated on a microchip. The valve is made with a 3D printer and is a simple structure that can be easily operated manually. A sample solution in a loading channel can be injected by switching the valve from the load to injection position. Sample injection tests using resorufin solutions revealed that samples can be injected below 100 µL min-1, and the performance of the sample injection system is comparable to that of a commercially available injector. In addition, the sample injection system was successfully applied to a flow-based assay for hydrogen peroxide. The detection limit (3σ) of hydrogen peroxide was estimated to be 0.5 µM, and the assay time after sample injection was approximately 100 s. The developed sample injection system will be useful for various microfluidic-based analyses including µFIA.

Differential Striatal Axonal Arborizations of the Intratelencephalic and Pyramidal-Tract Neurons: Analysis of the Data in the MouseLight Database.

There exist two major types of striatum-targeting neocortical neurons, specifically, intratelencephalic (IT) neurons and pyramidal-tract (PT) neurons. Regarding their striatal projections, it was once suggested that IT axons are extended whereas PT axons are primarily focal. However, subsequent study with an increased number of well-stained extended axons concluded that such an apparent distinction was spurious due to limited sample size. Recent work using genetically labeled neurons reintroduced the differential spatial extent of the striatal projections of IT and PT neurons through population-level analyses, complemented by observations of single axons. However, quantitative IT vs. PT comparison of a large number of axons remained to be conducted. We analyzed the data of axonal end-points of 161 IT neurons and 33 PT neurons in the MouseLight database (http://ml-neuronbrowser.janelia.org/). The number of axonal end-points in the ipsilateral striatum exhibits roughly monotonically decreasing distributions in both neuron types. Excluding neurons with no ipsilateral end-point, the distributions of the logarithm of the number of ipsilateral end-points are considerably overlapped between IT and PT neurons, although the proportion of neurons having more than 50 ipsilateral end-points is somewhat larger in IT neurons than in PT neurons. Looking at more details, among IT subpopulations in the secondary motor area (MOs), layer 5 neurons and bilateral striatum-targeting layer 2/3 neurons, but not contralateral striatum-non-targeting layer 2/3 neurons, have a larger number of ipsilateral end-points than MOs PT neurons. We also found that IT ipsilateral striatal axonal end-points are on average more widely distributed than PT end-points, especially in the medial-lateral direction. These results indicate that IT and PT striatal axons differ in the frequencies and spatial extent of end-points while there are wide varieties within each neuron type.

Drinking Levels and Profiles of Alcohol Addicted Rats Predict Response to Nalmefene.

Background: Pharmacotherapeutic options supporting the treatment of alcohol dependence are recommended and available but underutilized, partly due to questions about efficacy. Nalmefene, a µ-opioid receptor antagonist and partial kappa receptor agonist, is recommended for reduction of alcohol consumption, but evidence about its effectiveness has been equivocal; identifying factors which predict response will help optimize treatment. Methods: The alcohol deprivation effect paradigm is a tightly controlled procedure comprising repeated deprivation and reintroduction phases, leading to increased preference for alcohol; reintroduction approximates relapse. Using a digital drinkometer system measuring high-resolution drinking behavior, we examined the effects of nalmefene on relapse drinking behavior in alcohol addicted rats. We also tested whether drinking behavior in the relapse phase prior to nalmefene administration predicted treatment response. We further examined whether longitudinal drinking behavior and locomotor activity predicted treatment response. Results: Our results showed that nalmefene (0.3 mg/kg) reduced relapse-like consumption significantly (∼20%) compared to vehicle on the first 2 days of alcohol reintroduction. Examining the first 6 h of a preceded treatment-free relapse episode revealed drinking patterns clustering the rats into responders (reduction of >40%, n = 17) and non-responders (reduction of <40%, n = 7) to subsequent nalmefene treatment. During the first 6 h of the preceding relapse phase, responders consumed more alcohol than non-responders; the amount of alcohol consumed during each drinking approach was larger but frequency of drinking did not differ. Longitudinal drinking behavior and locomotor activity did not significantly predict response. Conclusion: Our results suggest that nalmefene reduces alcohol intake during a relapse-like situation but effectiveness can differ greatly at the individual level. However, who responds may be informed by examining drinking profiles and rats that show high drinking levels prior to treatment are more likely to respond to nalmefene.

Threshold-varying integrate-and-fire model reproduces distributions of spontaneous blink intervals.

Spontaneous blinking is one of the most frequent human behaviours. While attentionally guided blinking may benefit human survival, the function of spontaneous frequent blinking in cognitive processes is poorly understood. To model human spontaneous blinking, we proposed a leaky integrate-and-fire model with a variable threshold which is assumed to represent physiological fluctuations during cognitive tasks. The proposed model is capable of reproducing bimodal, normal, and widespread peak-less distributions of inter-blink intervals as well as the more common popular positively skewed distributions. For bimodal distributions, the temporal positions of the two peaks depend on the baseline and the amplitude of the fluctuating threshold function. Parameters that reproduce experimentally derived bimodal distributions suggest that relatively slow oscillations (0.11-0.25 Hz) govern blink elicitations. The results also suggest that changes in blink rates would reflect fluctuations of threshold regulated by human internal states.