From safety to frustration: The neural substrates of inhibitory learning in aversive and appetitive conditioning procedures.

Inhibitory associative learning counters the effects of excitatory learning, whether appetitively or aversively motivated. Moreover, the affective responses accompanying the inhibitory associations are of opponent valence to the excitatory conditioned responses. Inhibitors for negative aversive outcomes (e.g. shock) signal safety, while inhibitors for appetitive outcomes (e.g. food reward) elicit frustration and/or disappointment. This raises the question as to whether studies using appetitive and aversive conditioning procedures should demonstrate the same neural substrates for inhibitory learning. We review the neural substrates of appetitive and aversive inhibitory learning as measured in different procedural variants and in the context of the underpinning excitatory conditioning on which it depends. The mesocorticolimbic dopamine pathways, retrosplenial cortex and hippocampus are consistently implicated in inhibitory learning. Further neural substrates identified in some procedural variants may be related to the specific motivation of the learning task and modalities of the learning cues. Finally, we consider the translational implications of our understanding of the neural substrates of inhibitory learning, for obesity and addictions as well as for anxiety disorders.

Inhibitory Learning with Bidirectional Outcomes: Prevention Learning or Causal Learning in the Opposite Direction?

Influential models of causal learning assume that learning about generative and preventive relationships are symmetrical to each other. That is, a preventive cue directly prevents an outcome from occurring (i.e., "direct" prevention) in the same way a generative cue directly causes an outcome to occur. However, previous studies from our lab have shown that many participants do not infer a direct prevention causal structure after feature-negative discrimination (A+/AB-) with a unidirectional outcome (Lee & Lovibond, 2021). Melchers et al. (2006) suggested that the use of a bidirectional outcome that can either increase or decrease from baseline, encourages direct prevention learning. Here we test an alternative possibility that a bidirectional outcome encourages encoding of a generative relationship in the opposite direction, where B directly causes a decrease in the outcome. Thus, previous evidence of direct prevention learning using bidirectional outcomes may instead be explained by some participants inferring an "Opposite Causal" structure. In two experiments, participants did indeed report an opposite causal structure. In Experiment 1, these participants showed the lowest outcome predictions when B was combined with a novel cause in a summation test, and lowest outcome predictions when B was presented alone. In Experiment 2, B successfully blocked learning to a novel cue that was directly paired with a reduction in the outcome, and this effect was strongest among participants who endorsed an Opposite Causal structure. We conclude that previous evidence of direct prevention learning using bidirectional outcomes may be a product of excitatory rather than inhibitory learning.

Learning to stop responding.

Learning to stop responding is an important process that allows behavior to adapt to a changing and variable environment. This article reviews recent research in this laboratory and others that has studied how animals learn to stop responding in operant extinction, punishment, and feature-negative learning. Extinction and punishment are shown to be similar in two fundamental ways. First, the response-suppressing effects of both are highly context-specific. Second, the response-suppressing effects of both can be remarkably response-specific: Inhibition of one response transfers little to other responses. Learning to inhibit the response so specifically may result from the correction of "response error," the difference between the level of responding and what the current reinforcer supports. In contrast, the inhibition of responding that develops in feature-negative learning, where the response is reinforced during one discriminative stimulus (A) but not in a compound of A and stimulus B, is less response-specific: The inhibition of responding by stimulus B transfers and inhibits a second response, especially if the second response has itself been inhibited before. The results thus indicate both response-specific and response-general forms of behavioral inhibition. One possibility is that response-specific inhibition is learned when the circumstances encourage the organism to pay attention to the response-to what it is actually doing-as behavioral suppression is learned.

Second-Order Conditioning in Humans.

In contrast to the large body of work demonstrating second-order conditioning (SOC) in non-human animals, the evidence for SOC in humans is scant. In this review, I examine the existing literature and suggest theoretical and procedural explanations for why SOC has been so elusive in humans. In particular, I discuss potential interactions with conditioned inhibition, whether SOC is rational, and propose critical parameters needed to obtain the effect. I conclude that SOC is a real but difficult phenomenon to obtain in humans, and suggest directions for future research.

Second-Order Conditioning and Conditioned Inhibition in Different Moments of the Same Training: The Effect of A+ and AX- Trial Number.

The feature negative discrimination (A+/AX-) can result in X gaining excitatory properties (second-order conditioning, SOC) or in X gaining inhibitory properties (conditioned inhibition, CI), a challenging finding for most current associative learning theories. Research on the variables that modulate which of these phenomena would occur is scarce but has clearly identified the trial number as an important variable. In the set of experiments presented here, the effect of trial number was assessed in a magazine training task with rats as a function of both the conditioning sessions and the number of A+ and AX- trials per session, holding constant the total number of trials per session. The results indicated that SOC is most likely to be found at the beginning of training when there are many A+ and few AX- trials, and CI (as assessed by a retardation test) is most likely to be found at the end of training when there are few A+ and many AX- trials. Both phenomena were also found at different moments of training when the number of A+ trials was equal to the number of AX- trials. These results cannot be predicted by acquisition-focused associative models but can be predicted by theories that distinguish between learning and performance.

Inhibitory causal structures in serial and simultaneous feature negative learning.

We have previously reported that human participants trained with a simultaneous feature negative discrimination (intermixed A+/AB- trials) show only modest transfer of inhibitory properties of feature B to a separately trained excitor in a summation test. Their self-reported causal structure suggested that many participants learned that the effect of feature B was somewhat specific to the excitor it had been trained with (modulation), rather than learning that the feature prevented the outcome (prevention). This pattern is reminiscent of the distinction between negative occasion-setting and conditioned inhibition in the animal conditioning literature. However, in animals, occasion-setting is more commonly seen with a serial procedure, in which the feature (B) precedes the training excitor (A). Accordingly, we ran three experiments to compare serial with simultaneous training in an allergist causal judgement task. Transfer in a summation test was stronger to a previously modulated test excitor compared to a simple excitor after both simultaneous and serial training. There was a numerical trend towards a larger effect in the serial group, but it failed to reach significance and the Bayes Factor indicated support for the null. Serial training had no differential effect on the self-reported causal structure and did not significantly reduce overall transfer. After both simultaneous and serial training, transfer was strongest in participants who reported a prevention structure, replicating and extending our previous results to a previously modulated excitor. These results suggest that serial feature negative training does not promote a qualitatively different inhibitory causal structure compared to simultaneous training in humans.

Individual differences in causal structures inferred during feature negative learning.

Traditional associative learning theories predict that training with feature negative (A+/AB-) contingencies leads to the feature B acquiring negative associative strength and becoming a conditioned inhibitor (i.e., prevention learning). However, feature negative training can sometimes result in negative occasion setting, where B modulates the effect of A. Other studies suggest that participants learn about configurations of cues rather than their individual elements. In this study, we administered simultaneous feature negative training to participants in an allergist causal learning task and tested whether evidence for these three types of learning (prevention, modulation, configural) could be captured via self-report in the absence of any procedural manipulation. Across two experiments, we show that only a small subset of participants endorse the prevention option, suggesting that traditional associative models that predict conditioned inhibition do not completely capture how humans learn about negative contingencies. We also show that the degree of transfer in a summation test corresponds to the implied causal structure underlying conditioned inhibition, occasion-setting, and configural learning, and that participants are only partially sensitive to explicit hints about causal structure. We conclude that feature negative training is an ambiguous causal scenario that reveals individual differences in the representation of inhibitory associations, potentially explaining the modest group-level inhibitory effects often found in humans.

Fruit flies can learn non-elemental olfactory discriminations.

Associative learning allows animals to establish links between stimuli based on their concomitance. In the case of Pavlovian conditioning, a single stimulus A (the conditional stimulus, CS) is reinforced unambiguously with an unconditional stimulus (US) eliciting an innate response. This conditioning constitutes an 'elemental' association to elicit a learnt response from A+ without US presentation after learning. However, associative learning may involve a 'complex' CS composed of several components. In that case, the compound may predict a different outcome than the components taken separately, leading to ambiguity and requiring the animal to perform so-called non-elemental discrimination. Here, we focus on such a non-elemental task, the negative patterning (NP) problem, and provide the first evidence of NP solving in Drosophila. We show that Drosophila learn to discriminate a simple component (A or B) associated with electric shocks (+) from an odour mixture composed either partly (called 'feature-negative discrimination' A+ versus AB-) or entirely (called 'NP' A+B+ versus AB-) of the shock-associated components. Furthermore, we show that conditioning repetition results in a transition from an elemental to a configural representation of the mixture required to solve the NP task, highlighting the cognitive flexibility of Drosophila.

Stable Individual Differences in Occasion Setting.

In the current investigation, we classified participants as inhibitors or non-inhibitors depending on the extent to which they showed conditioned inhibition in a context that had been used for extinction of a conditioned response. This classification enabled us to predict participant responses in a second experiment which used a different design and a different experimental task. In the second experiment a feature-negative discrimination survived reversal training of the feature to a greater extent in the non-inhibitors than in the inhibitors and this result was supported by Bayesian analyses. We propose that the fundamental distinction between inhibitors and non-inhibitors is based on a tendency to utilize first-order (direct associations) or second-order (occasion-setting) strategies when faced with ambiguous information and that this classification is a stable individual differences attribute.

Can the discriminative stimulus effects of nicotine function concurrently as modulatory opponents in operant and pavlovian occasion setting paradigms in rats?

Nicotine promotes interoceptive changes in the nervous system. Such interoceptive stimuli play important roles in modulating addictive behavior. Operant and Pavlovian stimulus control modulate responsiveness to environmental stimuli related to drug-seeking and self-administration. Nicotine functions as a discriminative stimulus in modulating operant behavior as well as Pavlovian feature stimuli in modulating the conditional responding (CR) to exteroceptive CS→US contingencies. Elucidation of the interaction of these interoceptive stimulus control functions is vital for a comprehensive understanding of nicotine use/abuse, which might lead to better behavioral treatment strategies. This experiment evaluated the interaction among Pavlovian feature positive (FP) and feature negative (FN) effects of nicotine on concurrently occurring operant SD and SΔ effects. Sixteen rats were trained in a Pavlovian and operant bidirectional contingency paradigm, using nicotine (0.3 mg/kg) and non-drug (saline) states as interoceptive cues for operant discriminative stimulus conditions (SD and SΔ) as well as Pavlovian FP and FN for a light-CS, either leading to a shared food pellet outcome or non-outcome. Nicotine and saline sessions were intermixed. For one group of rats (n = 8), nicotine served as an SD for lever pressing (variable interval 60 s) and simultaneously functioned as an FN for CS-light→noUS relation on the same sessions. On intermixed sessions, saline served as the SΔ for lever pressing (non-reinforced) and FP, during which the 8-sec light preceded delivery of the food pellet (variable time ITI = 60 s). For the other group (n = 8) nicotine served as the SΔ (lever pressing non-reinforced) and FP for the CS, with saline serving in the reverse roles. Consecutive brief non-reinforcement tests revealed that: A) rates of lever pressing were significantly greater in SD than SΔ with nicotine and saline suggesting strong operant discriminative stimulus control; B) FP responding to the light CS with nicotine and saline was evident; and C) FN suppression of the CR with nicotine was not evident but weak under saline. These data suggest that nicotine can function as an interoceptive context that hierarchically can enter into concurrently opposing modulatory relations in Pavlovian and operant drug discrimination procedures.

The nicotine + alcohol interoceptive drug state: contribution of the components and effects of varenicline in rats.

RATIONALE: Nicotine and alcohol co-use is highly prevalent, and as such, individuals experience the interoceptive effects of both substances together. Therefore, examining sensitivity to a compound nicotine and alcohol (N + A) interoceptive cue is critical to broaden our understanding of mechanisms that may contribute to nicotine and alcohol co-use. OBJECTIVES: This work assessed the ability of a N + A interoceptive cue to gain control over goal-tracking behavior and determined the effects of the α4ß2 nicotinic partial agonist and smoking cessation compound varenicline on sensitivity to N + A. METHODS: Two groups of male Long Evans rats were trained to discriminate N + A (0.4 mg/kg nicotine + 1 g/kg alcohol, intragastric gavage (IG)) from water under two different training conditions using a Pavlovian drug discrimination task. The effects of varenicline (0, 1, 3 mg/kg, intraperitoneally (IP)) administered alone and on sensitivity to N + A and the components were determined. RESULTS: Under both training conditions, N + A rapidly gained control over behavior, with a greater contribution of nicotine to the N + A compound cue. Varenicline fully substituted for the N + A training dose, and varenicline (1 mg/kg) enhanced sensitivity to the lowest N + A dose (0.1 N + 0.1 A). Given the high selectivity of varenicline for the α4ß2 receptor, this finding suggests a functional role for α4ß2 nicotinic acetylcholine receptors (nAChRs) in modulating sensitivity to N + A. CONCLUSIONS: The N + A compound cue is a unique cue that is modulated, in part, by activity at the α4ß2 nAChR. These findings advance understanding of the interoceptive effects of nicotine and alcohol in combination and may have implications in relation to their co-use.

Behavioral inhibition during a conflict state elicits a transient decline in hippocampal theta power.

Although it has been shown that hippocampal theta power transiently declines during response inhibition in a simultaneous feature negative (FN: A+, AB-) task, observations of additional changes after this initial decline have been inconsistent across subjects. We hypothesized that the cause of these inconsistencies might be that variations in the learning speed for the FN task differentially affect the changes in hippocampal theta activity observed during the task. In this study, we classified rats into three groups (fast, intermediate, and slow FN-learning groups) based on the number of sessions required to complete learning of the FN task. We then examined whether there was a difference in hippocampal theta power among the fast, intermediate, and slow FN-learning groups, and rats that learned a simple discrimination task (SD group). We observed that compared to the SD group, the slow FN-learning group, but not the fast FN-learning group, showed an increase in hippocampal theta power. In addition, a transient decline of hippocampal theta power occurred in the fast FN-learning group, but not in the slow FN-learning group. These results indicate that the hippocampal theta activity during response inhibition in the FN task differed between fast- and slow-learning rats. Thus, we propose that a difference in learning speed affected hippocampal theta activity during response inhibition under a conflict state.

Hippocampal theta activity during behavioral inhibition for conflicting stimuli.

A recent behavioral inhibitory theory proposed that the hippocampus plays an important role in response inhibition to conflicting stimuli composed of simple inhibitory associations between events embedded in concurrent simple excitatory associations. In addition, the theory states that a serial feature negative (FN) task is a hippocampal-dependent task requiring the formation of a simple inhibitory association; on the other hand, a simple discrimination (SD) task is a typical hippocampus-independent task. In the present study, we recorded hippocampal theta activity from rats during FN and SD tasks to identify any potential differences. In the FN (A+, B→A-) task used in this study, rats were required to press a lever to present stimulus A (A+) and avoid pressing a lever to present a serial compound stimulus (B→A-). In the simple discrimination task (A+, B-), rats were required to press a lever to present stimulus A (A+) and avoid pressing a lever to present stimulus B (B-). We observed a transient decline of hippocampal theta power during response inhibition for a serial compound stimulus in the FN task. Thus, we conclude that the transient decline in hippocampal theta power reflects response inhibition for a conflicting stimulus. The results of the present study strongly support the behavioral inhibition theory.

Change in hippocampal theta activity with transfer from simple discrimination tasks to a simultaneous feature-negative task.

It was showed that solving a simple discrimination task (A+, B-) and a simultaneous feature-negative (FN) task (A+, AB-) used the hippocampal-independent strategy. Recently, we showed that the number of sessions required for a rat to completely learn a task differed between the FN and simple discrimination tasks, and there was a difference in hippocampal theta activity between these tasks. These results suggested that solving the FN task relied on a different strategy than the simple discrimination task. In this study, we provided supportive evidence that solving the FN and simple discrimination tasks involved different strategies by examining changes in performance and hippocampal theta activity in the FN task after transfer from the simple discrimination task (A+, B- → A+, AB-). The results of this study showed that performance on the FN task was impaired and there was a difference in hippocampal theta activity between the simple discrimination task and FN task. Thus, we concluded that solving the FN task uses a different strategy than the simple discrimination task.

The decline in rat hippocampal theta activity during response inhibition for the compound stimulus of negative patterning and simultaneous feature-negative tasks.

In experiment 1 of this study, we compared hippocampal theta activity between negative patterning and simple discrimination tasks. Our results demonstrated a transient decline in theta activity during response inhibition for a compound stimulus in the negative patterning task. In experiment 2 of this study, we compared hippocampal theta activity among simultaneous feature-negative, compound stimulus discrimination, and simple discrimination tasks in order to determine the cause of the decline in hippocampal theta activity during negative patterning tasks. Our results revealed that the decline in hippocampal theta activity occurred during the response inhibition for a compound stimulus in the simultaneous feature-negative task but not during the compound stimulus discrimination or simple discrimination tasks. Thus, we conclude that the transient decline in hippocampal theta activity is related to the inhibition in response to a compound stimulus that has an element that overlaps with a single stimulus.

Inter-relationships among diet, obesity and hippocampal-dependent cognitive function.

Intake of a Western diet (WD), which is high in saturated fat and sugar, is associated with deficits in hippocampal-dependent learning and memory processes as well as with markers of hippocampal pathology. In the present study, rats were trained to asymptote on hippocampal-dependent serial feature negative (FN) and hippocampal-independent simple discrimination problems. Performance was then assessed following 7 days on ad libitum chow and after 10, 24, 40, 60, and 90 days of maintenance on WD, on ketogenic (KETO) diet, which is high in saturated fat and low in sugar and other carbohydrates, or continued maintenance on chow (CHOW). Confirming and extending previous findings, diet-induced obese (DIO) rats fed WD showed impaired FN performance, increased blood-brain barrier (BBB) permeability, and increased fasting blood glucose levels compared to CHOW controls and to diet-resistant (DR) rats that did not become obese when maintained on WD. For rats fed the KETO diet, FN performance and BBB integrity were more closely associated with level of circulating ketone bodies than with obesity phenotype (DR or DIO), with higher levels of ketones appearing to provide a protective effect. The evidence also indicated that FN deficits preceded and predicted increased body weight and adiposity. This research (a) further substantiates previous findings of WD-induced deficits in hippocampal-dependent FN discriminations, (b) suggests that ketones may be protective against diet-induced cognitive impairment, and (c) provides evidence that diet-induced cognitive impairment precedes weight gain and obesity.