Early-extinction effects on a force response of humans.

Humans were used to investigate changes in response force occurring soon after reinforcement was eliminated. In Experiment 1, in a 300-s baseline phase, 10 participants received a point for holding down a pressure sensor set to operate at a force equal to 85% of the maximum force the participants exerted during a pretest. Following this, during a 600-s extinction phase, criterion responses had no consequence. In Experiment 2, 6 participants worked on the same task, but (a) points were exchangeable for money and (b) after extinction, the reinforcement baseline phase was reinstated. In Experiment 3, 6 participants completed the same task as in Experiment 2, but the required minimum force was 60% of the maximum force exerted during the pretest. In each experiment, increases in response force relative to the mean and peak force exerted during the last 100 s of baseline were observed in most participants when force responses were aggregated into short sample intervals, but less so with longer ones. The increases, however, were not systematic across or within participants, questioning the generality of and the criteria for demonstrating an extinction burst.

Context affects resurgence of negatively reinforced human behavior.

The effects of context on the resurgence of negatively reinforced (escape) responding was studied in an experiment with undergraduate students in which resurgence and renewal procedures were combined. Across conditions, in baseline (BL), key-pressing produced 3-s timeouts from pressing a force cell on a variable-ratio schedule of reinforcement; in the Alternative-Reinforcement and Test phases, a differential-reinforcement-of-other-behavior schedule and extinction were in effect, respectively. Conditions differed according to the context (the computer-screen color) in effect in each phase: ABA vs. ABB (with each letter representing a context; order of exposure to conditions was counterbalanced across participants). For each of six participants, independent of (a) order of exposure to conditions, (b) slight differences in BL reinforcement, and (c) differences in BL key-pressing rates, resurgence of greater magnitude occurred in the ABA than in the ABB condition. These results replicate and extend to contingencies of negative reinforcement previous findings with nonhumans and humans showing that context modulates the magnitude of resurgence.

Effects of reinforcement value on instruction following under schedules of negative reinforcement.

The effects of reinforcement value and social control on instruction following under a negative-reinforcement (escape) schedule were studied. Initially, responding produced timeouts from pressing a force cell under a low and a high force requirement on a fixed-ratio 1 schedule of reinforcement. Next, participants were reexposed to the low and high force requirements, but were instructed that the experimenter expected them to decrease the number of timeouts relative to the previous exposures to the procedure. Even though following the instruction led to a decrease in number of timeouts and to an increase in effort (i.e., was non-efficient), instruction following occurred consistently for each participant and was modulated by reinforcement value. That is, the decrease in the number of timeouts (i.e., instruction following) was lower under the high force requirement than under the low force requirement. These results replicate and extend previous findings that instructions interact with social and nonsocial contingencies in controlling human behavior.

The effects of instructions on the sensitivity of negatively reinforced human behavior to extinction.

The effects of instructions on the sensitivity of negatively reinforced (escape) behavior to extinction were studied. Initially, responding produced timeouts from pressing a force cell on a variable-ratio (VR) schedule, which was then discontinued (extinction). Based on extinction data, participants were distributed into two groups. Participants in the Persistence Group (for which response rates were low in extinction) were instructed that the experimenter expected them to continue responding in extinction after a second exposure to the VR schedule. Participants in the Extinction group (for which response rates were high in extinction) were instructed that the experimenter expected them to stop responding in extinction. Relative to the condition in which instructions were absent, extinction-response rates increased and decreased, respectively, for participants in the Persistence and Extinction groups. These results replicate and extend to negatively reinforced responding previous findings that showed behavioral control by instructions formulated as explicit experimenter demands or expectations.

The recurrence of negatively reinforced responding of humans.

The recurrence of negatively reinforced responding of humans was studied in three experiments. In each experiment during Baseline, key-pressing produced 3-s timeouts from a requirement to exert finger pressure on a force cell according to variable- or fixed-ratio schedules of reinforcement. In Experiment 1, resurgence was studied by arranging a differential-reinforcement-of-other-behavior schedule in the second phase, and extinction in the Test phase. In Experiment 2, ABA renewal was studied by extinguishing responding in the second phase in a different context and, in the Test phase, by presenting the Baseline-phase context when extinction still was in effect. In Experiment 3, reinstatement was studied by arranging extinction in the second phase, followed by the delivery of response-independent timeouts in the Test phase. Resurgence and renewal occurred consistently for each participant in Experiments 1 and 2, respectively. In Experiment 3, reinstatement was observed less consistently in four participants. The results of these experiments replicate and extend to negatively reinforced responding previous findings of the resurgence and renewal of positively reinforced responding obtained mainly with nonhuman animals.

Human choice under schedules of negative reinforcement.

The generalized matching equation provides a good description of response allocation in concurrent schedules of positive reinforcement in nonhumans as well as in humans. The present experiment was conducted to further investigate the allocation of responding under concurrent schedules of negative reinforcement (i.e., timeouts from pressing a force cell) in humans. Each of three participants was exposed to different reinforcement ratios (9:1, 1:1 and 1:9) in the terminal links of a concurrent-chains schedule of negative reinforcement. The allocation of responding under this schedule was well described by the generalized matching equation, for each participant. These results replicate previous findings obtained with nonhumans and humans under concurrent schedules of positive reinforcement. In addition, they extend the results reported by Alessandri and Rivière (2013) showing that human behavior maintained by timeouts from an effortful response is sensitive to changes in relative reinforcement ratios as well as relative delays of reinforcement.

Timeout from a high-force requirement as a reinforcer: an effective procedure for human operant research.

A procedure to study human operant conditioning is described using a timeout from a high-force requirement as reinforcer when a high force response was required. Experiment 1 reported evidence that a timeout from a high-force requirement acted as a reinforcer and a second experiment demonstrated sensitivity to delay to escape from the force requirement as a parameter of choice in a self-control paradigm. The results of the two experiments indicate a functional similarity between unconditioned reinforcers (e.g., food) used in nonhuman subjects and the present reinforcer, demonstrating that the present procedure is well-suited to study operant conditioning in humans.

Choices between segmented and unsegmented schedules and the self-control paradigm.

A concurrent-chains procedure was used to examine choice between a segmented (two- or three-terminal-link segments schedules) and an unsegmented schedule (simple schedule) in terminal links with equal interreinforcement intervals. In most such experiments, preference for the unsegmented schedule has been found, but in a recent study with humans (Alessandri et al., 2010) a reversal in preference was found when, in the segmented schedule, the terminal link segmenting stimulus was presented briefly and closer to food delivery such that the early terminal link stimulus was temporally closer to the food delivery. In Experiment 1, an attempt to replicate this latter effect with pigeons was unsuccessful but this outcome was consistent with an account in terms of a self-control contingency involving conditioned reinforcers. According to this account, the unsegmented alternative consisted of an immediate, smaller presentation of a conditioned reinforcer (i.e., the impulsive, and thus usually the preferred, option in several experiments) and the segmented schedule led to a delayed, larger conditioned reinforcer (i.e., the self-control option). In Experiment 2, a reversal of preference toward the segmented schedule was found when a delay was added to both terminal links between the reinforced initial-link response and the onset of the corresponding terminal link stimulus. This result is consistent with a similar effect found with primary reinforcers in the self-control literature suggesting the utility of self-control as an account of preferences for unsegmented terminal links of concurrent chains schedules.

Subjective time: cognitive and physical secondary tasks affect timing differently.

Humans were trained on a temporal discrimination to make one response when the stimulus duration was short (2 s) and a different response when the stimulus duration was long (8 s). They were then tested with stimulus durations in between to determine the bisection point. In Experiment 1, we examined the effect of a secondary cognitive task (counting backwards by threes) on the bisection point when participants were trained without a cognitive load and were tested with a cognitive load or the reverse (relative to appropriate controls). When the cognitive load increased from training, the psychophysical function plotting long responses against the increase in stimulus duration shifted to the right (as if the internal clock slowed down), and when the cognitive load decreased from training the psychophysical function shifted to the left (as if the internal clock speeded up). In Experiment 2, when the secondary task consisted of exerting continuous force on a transducer (a physically effortful task), it had the opposite effect. When the required force increased from training, the psychophysical function shifted to the left (as if the internal clock speeded up), and when the required force decreased from training, the psychophysical function shifted to the right (as if the internal clock slowed down). The results support an attentional view of the subjective passage of time. A cognitive secondary task appears to decrease attention to temporal cues, resulting in the underestimation of the passage of time, whereas a force requirement appears to increase attention to temporal cues, resulting in the overestimation of the passage of time.

Preference for a segmented schedule using a brief S+ stimulus correlated with a great delay reduction in humans.

Past studies using the concurrent-chain procedure showed that pigeons and humans generally prefer an unsegmented schedule to a segmented schedule. This finding is ostensibly inconsistent with theories of conditioned reinforcement such as delay-reduction theory. In the present study with humans, two changes in the basic segmented schedule were implemented to resolve this inconsistency. The first change was that in the segmented schedule the terminal-link stimulus (S+ stimulus) changed late in the terminal-link, close to reinforcement presentation. The second change was that the presentation of the segmenting stimulus, S+, was brief allowing a reinstatement of the early terminal-link stimulus, which is contiguous with reinforcement. Our data constitute the first demonstration of preference for the segmented schedule when a brief S+ is correlated with a greater reduction in delay to reinforcement.

Preference for 50% reinforcement over 75% reinforcement by pigeons.

When pigeons are given a choice between an initial-link alternative that results in either a terminal-link stimulus correlated with 100% reinforcement or a stimulus correlated with 0% reinforcement (overall 50% reinforcement) and another initial-link alternative that always results in a terminal-link stimulus correlated with 100% reinforcement, some pigeons show a preference for the initial-link alternative correlated with 50% reinforcement. Using this procedure, in Experiment 1, we found a relatively modest preference for 100% over 50% reinforcement. In Experiment 2, we decreased the reinforcement density for the second initial-link alternative to 75% and found a significant preference for the 50% reinforcement initial-link alternative. It may be that this "maladaptive" behavior results from a positive contrast between the expectation of reinforcement correlated with the 50% reinforcement initial-link alternative and the terminal-link stimulus correlated with 100% reinforcement. But apparently, the complementary negative contrast does not develop between the expectation of reinforcement correlated with the 50% reinforcement initial-link alternative and the terminal-link stimulus correlated with 0% reinforcement that often follow. Such paradoxical choice may account for certain human appetitive risk-taking behavior (e.g., gambling) as well.

Within-trial contrast: The effect of probability of reinforcement in training.

There is evidence that pigeons prefer conditioned reinforcers that are preceded by greater effort over those that are preceded by less effort (an effect that has been attributed to within-trial contrast). In past research the probability of reinforcement for correct choice of the conditioned reinforcer has been 100%, however, the high level of reinforcement for both alternatives in training may result in a performance ceiling when choice between those alternatives is provided on test trials. In the present study we tested this hypothesis by including a group for which the probability of reinforcement in training was only 50%. Pigeons were trained on two simultaneous discriminations, one that was preceded by a 30 peck requirement the other by a single peck requirement. On test trials, we found a significant preference for the S+ that required the greater effort in training for pigeons trained with 100% and a small but nonsignificant effect for pigeons trained with 50% reinforcement. Although the hypothesis that the within-trial contrast effect was constrained by a performance ceiling was not confirmed, we did find a reliable within-trial contrast effect with 100% reinforcement.

Preference for rewards that follow greater effort and greater delay.

Humans prefer (conditioned) rewards that follow greater effort (Aronson & Mills, 1959). This phenomenon can be interpreted as evidence for cognitive dissonance (or as justification of effort) but may also result from (1) the contrast between the relatively greater effort and the signal for reinforcement or (2) the delay reduction signaled by the conditioned reinforcer. In the present study, we examined the effect of prior force and prior time to produce stimuli associated with equal reinforcement. As expected, pressing with greater force or for a longer time was less preferred than pressing with less force or for a shorter time. However, participants preferred the conditioned reinforcer that followed greater force and more time. Furthermore, participants preferred a long duration with no force requirement over a shorter duration with a high force requirement and, consistent with the contrast account but not with the delay reduction account, they preferred the conditioned stimulus that followed the less preferred, shorter duration, high-force event. Within-trial contrast provides a more parsimonious account than justification of effort, and a more complete account than delay reduction.

Cognitive dissonance in children: justification of effort or contrast?

Justification of effort is a form of cognitive dissonance in which the subjective value of an outcome is directly related to the effort that went into obtaining it. However, it is likely that in social contexts (such as the requirements for joining a group) an inference can be made (perhaps incorrectly) that an outcome that requires greater effort to obtain in fact has greater value. Here we present evidence that a cognitive dissonance effect can be found in children under conditions that offer better control for the social value of the outcome. This effect is quite similar to contrast effects that recently have been studied in animals. We suggest that contrast between the effort required to obtain the outcome and the outcome itself provides a more parsimonious account of this phenomenon and perhaps other related cognitive dissonance phenomena as well. Research will be needed to identify cognitive dissonance processes that are different from contrast effects of this kind.