Relapse after intermittent access to cocaine: Discriminative cues more effectively trigger drug seeking than do conditioned cues.

RATIONALE: When people with drug addiction encounter cues associated with drug use, this can trigger cravings and relapse. These cues can include conditioned stimuli (CSs) signaling drug delivery and discriminative stimuli (DSs) signaling drug availability. Compared to CS effects, DS effects are less explored in preclinical studies on cue-induced relapse. OBJECTIVE: We compared CS and DS effects on reward seeking following abstinence from intermittent-access cocaine (or sucrose) self-administration. METHODS: During 15-20 intermittent-access sessions, rats self-administered i.v. cocaine or sucrose pellets paired with a light-tone CS. Cocaine/sucrose was available for 5-min (signalled by a light; DS+) and unavailable for 25 min (signalled by different lighting conditions; DS-), and this cycled for 4 h/session. Following abstinence, we measured cocaine/sucrose seeking under extinction triggered by CS and DS presentation, and instrumental responding reinforced by these cues. RESULTS: Across intermittent-access sessions, rats increased lever pressing for cocaine or sucrose during DS+ periods and decreased responding during DS- periods. On days 2 and 21 of abstinence, only presentation of the DS+ or DS+ and CS combined elicited increased cocaine/sucrose-seeking behaviour (i.e., increased active lever presses). Presenting the DS- alongside the DS+ suppressed the increased cocaine-seeking behaviour otherwise produced by the DS+ . Finally, on day 21 of abstinence, rats showed equivalent levels of lever pressing reinforced by the DS+ , CS and by the DS+ and CS combined, suggesting comparable conditioned reinforcing value. CONCLUSIONS: After intermittent self-administration, cocaine-associated DSs and CSs acquire similar conditioned reinforcing properties, but DSs more effectively trigger increases in drug seeking.

Temporal context effects on suboptimal choice.

Choice can be driven both by rewards and stimuli that signal those rewards. Under certain conditions, pigeons will prefer options that lead to less probable reward when the reward is signaled. A recently quantified model, the Signal for Good News (SiGN) model, assumes that in the context of uncertainty, signals for a reduced delay to reward reinforce choice. The SiGN model provides an excellent fit to previous results from pigeons and the current studies are the first to test a priori quantitative predictions. Pigeons chose between a suboptimal alternative that led to signaled 20% food and an optimal alternative that led to 50% food. The duration of the choice period was manipulated across conditions in two experiments. Pigeons strongly preferred the suboptimal alternative at the shorter durations and strongly preferred the optimal alternative at the longer durations. The results from both experiments fit well with predictions from the SiGN model and show that altering the duration of the choice period has a dramatic effect in that it changes which of the two options pigeons prefer. More generally, these results suggest that the relative value of options is not fixed, but instead depends on the temporal context.

Evaluation of a discrimination training procedure for establishing praise as a reinforcer.

We evaluated the effects of a discrimination training procedure for establishing praise as a reinforcer for three children with autism spectrum disorder. After establishing two praise words as discriminative stimuli and two nonsense words as S-deltas, we evaluated whether the stimuli then functioned as reinforcers by presenting each stimulus as a consequence for a new response. The results demonstrated that previously neutral praise words functioned as reinforcers and nonsense words did not. As in previous studies on establishing reinforcers, the effects were transitory, and praise words did not continue to function as reinforcers after repeated exposure without discrimination training. Recommendations are provided for future research and maintaining reinforcement effects.

The effects of extinction and an explicitly unpaired treatment on the reinforcing properties of a Pavlovian conditioned stimulus.

This series of experiments examined the effects of extinction and an explicitly unpaired treatment on the ability of a conditioned stimulus (CS) to function as a reinforcer. Rats were trained to lever press for food, exposed to pairings of a noise CS and food, and, finally, tested for their willingness to lever press for the CS in the absence of the food. Experiment 1 provided a demonstration of conditioned reinforcement (using controls that were only exposed to unpaired presentations of the CS and food) and showed that it was equivalent after one or four sessions of CS-food pairings. Experiments 2 and 3 showed that, after one session of CS-food pairings, repeated presentations of the CS alone reduced its reinforcing properties; but after four sessions of CS-food pairings, repeated presentations of the CS alone had no effect on these properties. Experiment 4 showed that, after four sessions of CS-food pairings, explicitly unpaired presentations of the CS and food completely undermined conditioned reinforcement. Finally, Experiment 5 provided within-experiment evidence that, after four sessions of CS-food pairings, the reinforcing properties of the CS were disrupted by explicitly unpaired presentations of the CS and food but spared by repeated presentations of the CS alone. Together, these findings indicate that the effectiveness of extinction in undermining the reinforcing properties of a CS depends on its level of conditioning; and that, where extinction fails to disrupt these properties, they are successfully undermined by an explicitly unpaired treatment. They are discussed with respect to findings in the literature on Pavlovian-to-instrumental transfer; and the Rescorla-Wagner model, which anticipates that an explicitly unpaired treatment will be more effective than extinction in reversing the effects of conditioning.

Cue- versus reward-encoding basolateral amygdala projections to nucleus accumbens.

In substance use disorders, drug use as unconditioned stimulus (US) reinforces drug taking. Meanwhile, drug-associated cues (conditioned stimulus [CS]) also gain incentive salience to promote drug seeking. The basolateral amygdala (BLA) is implicated in both US- and CS-mediated responses. Here, we show that two genetically distinct BLA neuronal types, expressing Rspo2 versus Ppp1r1b, respectively, project to the nucleus accumbens (NAc) and form monosynaptic connections with both dopamine D1 and D2 receptor-expressing neurons. While intra-NAc stimulation of Rspo2 or Ppp1r1b presynaptic terminals establishes intracranial self-stimulation (ICSS), only Ppp1r1b-stimulated mice exhibit cue-induced ICSS seeking. Furthermore, increasing versus decreasing the Ppp1r1b-to-NAc, but not Rspo2-to-NAc, subprojection increases versus decreases cue-induced cocaine seeking after cocaine withdrawal. Thus, while both BLA-to-NAc subprojections contribute to US-mediated responses, the Ppp1r1b subprojection selectively encodes CS-mediated reward and drug reinforcement. Such differential circuit representations may provide insights into precise understanding and manipulation of drug- versus cue-induced drug seeking and relapse.

Effects of inter-trial interval on sign-tracking and conditioned reinforcer efficacy in female rats.

Previous nonhuman studies have reported that sign-tracking to a conditioned stimulus (CS) is increased when the intertrial interval (ITI) duration is increased. Separate studies indicate that individual differences in sign-tracking (vs. goal-tracking) at a fixed ITI (and CS duration) is predictive of the conditioned reinforcer efficacy of the CS. The present study evaluates, for the first time, if increasing the ITI increases rats' sign-tracking and the conditioned reinforcing efficacy of the CS. Forty-five female rats were randomly assigned to one of three groups that completed appetitive Pavlovian training with ITIs of 14, 24, or 96 s. Subsequently, they completed tests of conditioned reinforcement. Replicating previous findings, longer ITIs increased sign-tracking to a lever-CS and, extending the literature, conditioned reinforcer efficacy of that CS was highest at the longest ITI used during Pavlovian training. Implications for behavioral interventions using conditioned reinforcement are discussed.

More complex incidental bidirectional naming results from exposure alone.

Incidental bidirectional naming (Inc-BiN) has been defined as a verbal developmental cusp whereby children demonstrate learning the names of things as listener and speaker as a function of observation alone. Stimulus characteristics have been found to affect performance in tests for Inc-BiN. To further explore this effect, Experiment 1 compared untaught listener and speaker responses for novel familiar-type versus novel nonfamiliar-type stimuli with 20 first-grade students following naming experiences in which the participants observed each visual stimulus five times while hearing its name. Participants performed significantly better with familiar-type than with nonfamiliar-type stimuli. Experiment 2 examined the effects of a repeated-probe intervention to induce Inc-BiN with nonfamiliar-type stimuli. Participants were six first-grade students who demonstrated incidental unidirectional naming (i.e., acquired names as listener from exposure alone). Implementation of the intervention was staggered across dyads of participants in a multiple-probe, simultaneous-treatments design. One participant in each dyad received the intervention with nonfamiliar-type stimuli only and the other with both nonfamiliar- and familiar-type stimuli. Pre- and postintervention Inc-BiN probes with stimuli not included in the intervention suggested both conditions were effective in establishing Inc-BiN for nonfamiliar-type stimuli. These findings have implications for understanding the mechanisms underlying Inc-BiN.

Models of conditioned reinforcement and abnormal behaviour in captive animals.

Abnormal behaviours are common in captive animals, and despite a lot of research, the development, maintenance and alleviation of these behaviours are not fully understood. Here, we suggest that conditioned reinforcement can induce sequential dependencies in behaviour that are difficult to infer from direct observation. We develop this hypothesis using recent models of associative learning that include conditioned reinforcement and inborn facets of behaviour, such as predisposed responses and motivational systems. We explore three scenarios in which abnormal behaviour emerges from a combination of associative learning and a mismatch between the captive environment and inborn predispositions. The first model considers how abnormal behaviours, such as locomotor stereotypies, may arise from certain spatial locations acquiring conditioned reinforcement value. The second model shows that conditioned reinforcement can give rise to abnormal behaviour in response to stimuli that regularly precede food or other reinforcers. The third model shows that abnormal behaviour can result from motivational systems being adapted to natural environments that have different temporal structures than the captive environment. We conclude that models including conditioned reinforcement offer an important theoretical insight regarding the complex relationships between captive environments, inborn predispositions, and learning. In the future, this general framework could allow us to further understand and possibly alleviate abnormal behaviours.

Pavlovian learning and conditioned reinforcement.

Conditioned reinforcers are widely used in applied behavior analysis. Basic research evidence reveals that Pavlovian learning plays an important role in the acquisition and efficacy of new conditioned-reinforcer functions. Thus, a better understanding of Pavlovian principles holds the promise of improving the efficacy of conditioned reinforcement in applied research and practice. This paper surveys how (and if) Pavlovian principles are presented in behavior-analytic textbooks; imprecisions and knowledge gaps within contemporary Pavlovian empirical findings are highlighted. Thereafter, six practical principles of Pavlovian conditioning are presented along with empirical support and knowledge gaps that should be filled by applied and translational behavior-analytic researchers. Innovative applications of these principles are outlined for research in language acquisition, token reinforcement, and self-control.

An evaluation of methods for studying the effects of conditioned reinforcement on human choice.

Shahan et al. (2006) found that the relative rate of pigeons' pecking on two observing responses (i.e., responses that only produced an S+ or stimulus correlated with primary reinforcement) was well described by the relative rate of S+ delivery. Researchers have not evaluated the effects of S+ delivery rate in a concurrent observing response procedure with human subjects, so the necessary procedural modifications for studying the effects of conditioned reinforcement on human choice remain unclear. The purpose of the current study was to conduct an additive component analysis of modifications to the procedures of Shahan et al. (2006). We evaluated the additive effects of introducing response cost, a changeover response, and ordinal discriminative stimuli on correspondence with the results of Shahan et al. and the quality of fits of the generalized matching equation. When our procedures were most similar to those of Shahan et al., we observed low rates of observing and indifference between the two observing responses. For the group of subjects with whom all three additive components were included, we obtained the highest level of sensitivity to relative rate of S+ delivery, but the slope and R2 of our fits of the generalized matching equation were still much lower than those obtained by Shahan et al. Potential reasons for these discrepancies, methods of resolving them, and implications for future research are discussed.

Optogenetic activation of basolateral amygdala-to-nucleus accumbens core neurons promotes Pavlovian approach responses but not instrumental pursuit of reward cues.

Reward-associated conditioned stimuli (CSs) can acquire predictive value, evoking conditioned approach behaviours that prepare animals to engage with forthcoming rewards. Such CSs can also acquire conditioned reinforcing value, becoming attractive and pursued. Through their conditioned effects, CSs can promote adaptive (e.g., locating food) but also maladaptive behaviours (e.g., drug use). Basolateral amygdala neurons projecting to the nucleus accumbens core (BLA→NAc core neurons) mediate the response to appetitive CSs, but the extent to which this involves effects on the predictive and/or conditioned reinforcing properties of CSs is unclear. Thus, we examined the effects of optogenetic stimulation of BLA→NAc core neurons on i) CS-triggered approach to the site of reward delivery, a Pavlovian conditioned approach response and ii) the instrumental pursuit of a CS, a measure of conditioned reinforcement. Water-restricted, adult male rats learned that a light-tone compound cue (the CS) predicts water delivery into a receptacle. Pairing optogenetic stimulation of BLA→NAc core neurons with CS presentation potentiated CS-triggered water receptacle visits. This suggests that activity in BLA→NAc core neurons promotes Pavlovian goal-approach behaviour. Next, rats could lever press for CS presentations, without water delivery. Optogenetic stimulation of BLA→NAc core neurons either during instrumental test sessions or during prior CS-water conditioning did not influence lever responding for the CS. This suggests that activity in BLA→NAc core neurons does not influence the instrumental pursuit of a water-paired CS. We conclude that activation of BLA→NAc core neurons promotes cue-induced control over behaviour by increasing conditioned goal-approach responses, without affecting the operant pursuit of reward cues.

Muscarinic and NMDA Receptors in the Substantia Nigra Play a Role in Reward-Related Learning.

BACKGROUND: Reward-related learning, where animals form associations between rewards and stimuli (i.e., conditioned stimuli [CS]) that predict or accompany those rewards, is an essential adaptive function for survival. METHODS: In this study, we investigated the mechanisms underlying the acquisition and performance of conditioned approach learning with a focus on the role of muscarinic acetylcholine (mACh) and NMDA glutamate receptors in the substantia nigra (SN), a brain region implicated in reward and motor processes. RESULTS: Using RNAscope in situ hybridization assays, we found that dopamine neurons of the SN express muscarinic (mACh5), NMDA2a, NMDA2b, and NMDA2d receptor mRNA but not mACh4. NMDA, but not mACh5, receptor mRNA was also found on SN GABA neurons. In a conditioned approach paradigm, rats were exposed to 3 or 7 conditioning sessions during which light/tone (CS) presentations were paired with delivery of food pellets, followed by a test session with CS-only presentations. Intra-SN microinjections of scopolamine (a mACh receptor antagonist) or AP-5 (a NMDA receptor antagonist) were made either prior to each conditioning session (to test their effects on acquisition) or prior to the CS-only test (to test their effects on expression of the learned response). Scopolamine and AP-5 produced dose-dependent significant reductions in the acquisition, but not performance, of conditioned approach. CONCLUSIONS: These results suggest that SN mACh and NMDA receptors are key players in the acquisition, but not the expression, of reward-related learning. Importantly, these findings redefine the role of the SN, which has traditionally been known for its involvement in motor processes, and suggest that the SN possesses attributes consistent with a function as a hub of integration of primary reward and CS signals.

Paradoxical choice and the reinforcing value of information.

Signals that reduce uncertainty can be valuable because well-informed decision-makers can better align their preferences to opportunities. However, some birds and mammals display an appetite for informative signals that cannot be used to increase returns. We explore the role that reward-predictive stimuli have in fostering such preferences, aiming at distinguishing between two putative underlying mechanisms. The 'information hypothesis' proposes that reducing uncertainty is reinforcing per se, somewhat consistently with the concept of curiosity: a motivation to know in the absence of tractable extrinsic benefits. In contrast, the 'conditioned reinforcement hypothesis', an associative account, proposes asymmetries in secondarily acquired reinforcement: post-choice stimuli announcing forthcoming rewards (S+) reinforce responses more than stimuli signalling no rewards (S-) inhibit responses. In three treatments, rats faced two equally profitable options delivering food probabilistically after a fixed delay. In the informative option (Info), food or no food was signalled immediately after choice, whereas in the non-informative option (NoInfo) outcomes were uncertain until the delay lapsed. Subjects preferred Info when (1) both outcomes were explicitly signalled by salient auditory cues, (2) only forthcoming food delivery was explicitly signalled, and (3) only the absence of forthcoming reward was explicitly signalled. Acquisition was slower in (3), when food was not explicitly signalled, showing that signals for positive outcomes have a greater influence on the development of preference than signals for negative ones. Our results are consistent with an elaborated conditioned reinforcement account, and with the conjecture that both uncertainty reduction and conditioned reinforcement jointly act to generate preference.

Some detrimental effects of conditioned reinforcement on the maintenance of dog behavior.

In animal training, there is disagreement regarding whether a conditioned reinforcer, such as the sound of a clicker, should always be followed by an unconditioned reinforcer, such as a treat. Proponents for clicking without always giving a treat argue that the click can substitute for the reinforcer, due to Pavlovian conditioning and the partial reinforcement effect. Those who advise always following the click with a treat argue that the clicker will become an unreliable predictor of food if it is not always followed by an unconditioned reinforcer. In this study, a within-subject reversal design with two dogs was used to compare the behavioral effects of always following a click with food (one click condition) and only sometimes following a click with food (two clicks condition). Results showed that the two clicks condition disrupted the frequency, accuracy, and topography of the behavior and increased noncompliance and other unwanted behaviors. While the detrimental effects of this condition may seem paradoxical at first, they can be explained by the discriminative properties of the conditioned reinforcer.

A Preliminary Analysis of the Effects of Clicker Training and Verbal Instructions on the Acquisition of Relationship-Building Skills in Two Applied Behavior Analysis Practitioners.

Recent research has emphasized the need for training and competency beyond the standard technical skills acquired by applied behavior analysis (ABA) practitioners, including essential relationship-building and compassionate care skills (Taylor et al., 2019). Clicker training is a well-established behavior-analytic method for improving performance via immediate feedback in the form of an audible "click." The effectiveness of clicker training has not yet been evaluated as a technique for shaping complex clinical repertoires. This study evaluated the effects of verbal instructions, clicker training, and role-play on the acquisition of therapeutic relationship skills in ABA practitioners. Data were obtained as part of a training program conducted within an ABA agency, and the acquisition of target skills was evaluated using a multiple-baseline design across behaviors for two participants. During baseline, participants rarely demonstrated target skills. During training, the procedure resulted in increased engagement in all three target skills for both participants. Skill generalization with respect to untrained and novel scenarios was observed but at levels below mastery. Findings have potential implications for trainers and supervisors seeking efficient, nonintrusive, socially acceptable methods of improving practitioner performance.

Forced-exposure trials increase suboptimal choice.

The influence of single option or forced-exposure (FE) trials was studied in the suboptimal choice task. Pigeons chose between an optimal alternative that led to food half of the time and a suboptimal alternative that led to food 20% of the time. Choice of the suboptimal alternative was compared across groups of subjects that received different numbers of FE trials during training. In Experiment 1, subjects received 100% FE trials, 67% FE trials, or only choice trials. Pigeons in the two groups that had FE trials developed extreme preference for the signaled suboptimal alternative over the unsignaled optimal alternative, while pigeons that had no FE trials showed pronounced individual differences. Experiment 2 compared 10% and 90% FE trials. When neither alternative signaled trial outcomes, both groups of subjects strongly preferred the optimal alternative. When the suboptimal alternative provided differential signals, the subjects in the 90% FE group developed strong preference for the suboptimal alternative and subjects in the 10% FE group maintained preference for the optimal alternative. The results of both experiments demonstrate that FE trials can have substantial effects on the development of preference in the suboptimal choice task.

Good news is better than bad news, but bad news is not worse than no news.

Under certain conditions, pigeons will reliably prefer an alternative that leads to a lower probability of food over an alternative that leads to a higher probability of food (i.e., demonstrate suboptimal choice). A critical aspect of the typical procedure is that the alternative associated with less food provides differential stimuli that signal trial outcomes, but the alternative associated with more food does not. Few studies have investigated how partial signaling of an alternative influences preference. In Experiments 1-3, pigeons chose between two alternatives that each led to food 60% of the time with partially signaled trial outcomes. One alternative occasionally provided a stimulus that always preceded food (i.e., "good news") and the other alternative occasionally provided a stimulus that always preceded no food ("bad news"). Experiments 2 and 3 also assessed preference in conditions in which alternatives were either completely unsignaled (provided no differential stimuli) or always led to food. Pigeons consistently preferred the "good news" alternative over the "bad news" alternative and preferred 100% food over the "bad news" alternative. The results from conditions in which pigeons chose between the "bad news" alternative and an unsignaled alternative were inconclusive, but suggestive of a preference for bad news. The results are used to evaluate and distinguish between competing explanations of suboptimal choice.

Behavioral ephemera, difficult discriminations, and behavioral stability.

Every species possesses abilities for successfully interacting with its environment. These result from phylogeny. In the laboratory, one may arrange artificial conditions that thwart an organism's abilities. The result may be a "phenomenon." With sufficient training, however, the phenomenon may prove to be ephemeral, as the organism's basic abilities reassert themselves. Pigeons respond extremely well to differences and nondifferences in rate of obtaining food. This ability may be thwarted in a variety of ways, but the results tend to be ephemeral. An example is an experiment that pitted pigeons' preference for unimpeded responding against their ability to respond to food rate. In a concurrent-chains procedure, the terminal links were identical variable-interval schedules, but in one terminal link, every response produced a timeout. The duration of the timeout varied, and preference varied with it, but the relation vanished with training, in keeping with the equality of food rate across the 2 terminal links. Some other examples of "phenomena" that tend to disappear with sufficient training and sufficient variation in experimental parameters are behavioral contrast, conditioned reinforcement, resistance to extinction, and suboptimal choice. These "phenomena" depend on pigeons' failing to make difficult discriminations. They appear to be behavioral ephemera.

The Effect of the Establishment of Conditioned Reinforcement for Reading Content on Second-Graders' Reading Achievement.

There remains a gap in the current literature as to how to reliably measure and increase students' "voluntary reading," based on research suggesting a relation between reading amount and reading achievement. We tested the effect of the establishment of conditioned reinforcement for reading via a collaborative shared reading (CSR) conditioning procedure on eight 2nd-grade students with and without learning disabilities and developmental disorders. This conditioning procedure was composed of opportunities for reciprocal reading and collaboration on comprehension and vocabulary tasks related to the reading content, such that partners (teacher-participant or participant-participant) were required to work together. We utilized a combined small-n experimental-control simultaneous-treatment design with a single-case multiple-probe design nested within each small group in order to compare within- and between-group differences for participants in the CSR procedure with a teacher or peer. All participants for whom conditioned reinforcement for reading was established (n = 7) demonstrated gains in reading achievement after a maximum of nine sessions (412 min), with grade-level increases between 0.2 and 2.5 on measures of reading comprehension and between 0.3 and 3.1 on measures of vocabulary. The students in the teacher-yoked condition (n = 3) demonstrated more significant gains in their average increases in achievement, although the peer-yoked procedure was also effective and possibly more viable in a classroom setting. These results suggest that a CSR procedure with a teacher or peer should be considered as a means of increasing the reading achievement of early elementary students via increases in the reinforcement value of reading.

Devaluation of a conditioned reinforcer requires its reexposure.

A change in motivational state does not guarantee a change in operant behaviour. Only after an organism has had contact with an outcome while in a relevant motivational state does behaviour change, a phenomenon called incentive learning. While ample evidence indicates that this is true for primary reinforcers, it has not been established for conditioned reinforcers. We performed an experiment with rats where lever-presses were reinforced by presentations of an audiovisual stimulus that had previously preceded food delivery; in the critical experimental groups, the audiovisual stimulus was then paired a single time with a strong electric shock. Some animals were reexposed to the audiovisual stimulus. Lever-presses yielding no outcomes were recorded in a subsequent test. Animals that had been reexposed to the audiovisual stimulus after the aversive training responded less than did those that had not received reexposure. Indeed, those animals that were not reexposed did not differ from a control group that received no aversive conditioning of the audiovisual stimulus. Moreover, these results were not mediated by a change in the food's reinforcement value, but instead reflect a change in behaviour with respect to the conditioned reinforcer itself. These are the first data to indicate that the affective value of conditioned stimuli, like that of unconditioned ones, is established when the organism comes into contact with them.