Temporal context effects on suboptimal choice.

Choice can be driven both by rewards and stimuli that signal those rewards. Under certain conditions, pigeons will prefer options that lead to less probable reward when the reward is signaled. A recently quantified model, the Signal for Good News (SiGN) model, assumes that in the context of uncertainty, signals for a reduced delay to reward reinforce choice. The SiGN model provides an excellent fit to previous results from pigeons and the current studies are the first to test a priori quantitative predictions. Pigeons chose between a suboptimal alternative that led to signaled 20% food and an optimal alternative that led to 50% food. The duration of the choice period was manipulated across conditions in two experiments. Pigeons strongly preferred the suboptimal alternative at the shorter durations and strongly preferred the optimal alternative at the longer durations. The results from both experiments fit well with predictions from the SiGN model and show that altering the duration of the choice period has a dramatic effect in that it changes which of the two options pigeons prefer. More generally, these results suggest that the relative value of options is not fixed, but instead depends on the temporal context.

Suboptimal choice: A review and quantification of the signal for good news (SiGN) model.

As first reported several decades ago, pigeons (Columba livia) sometimes choose options that provide less food over options that provide more food. This behavior has been variously referred to as suboptimal, maladaptive, or paradoxical because it lowers overall food intake. A great deal of research has been directed at understanding the conditions under which animals and people make suboptimal choices and the mechanisms that drive this behavior. Here, we review the literature on suboptimal choice and the variables that play a role in this phenomenon. Suboptimal choice is most likely to occur when the outcomes following a choice are uncertain, when the outcomes are delayed after the choice, and when the outcomes are signaled only on the option that provides food less often. We propose a mathematical formalization of the signal for good news (SiGN) model which assumes that a signal for a reduction in delay to food reinforces choice. We generate predictions from the model about the effect of parameters that characterize suboptimal choice and we show that, even in the absence of free parameters, the SiGN model provides a very good fit to the choice proportions of birds from a large set of conditions across studies from numerous researchers. R code for SiGN predictions and the data set are available on the Open Science Framework (https://osf.io/39qtj). We discuss limitations of the model, propose directions for future research, and discuss the general applicability of this research to understanding how rewards and signals for reward may combine to reinforce behavior. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Forced-exposure trials increase suboptimal choice.

The influence of single option or forced-exposure (FE) trials was studied in the suboptimal choice task. Pigeons chose between an optimal alternative that led to food half of the time and a suboptimal alternative that led to food 20% of the time. Choice of the suboptimal alternative was compared across groups of subjects that received different numbers of FE trials during training. In Experiment 1, subjects received 100% FE trials, 67% FE trials, or only choice trials. Pigeons in the two groups that had FE trials developed extreme preference for the signaled suboptimal alternative over the unsignaled optimal alternative, while pigeons that had no FE trials showed pronounced individual differences. Experiment 2 compared 10% and 90% FE trials. When neither alternative signaled trial outcomes, both groups of subjects strongly preferred the optimal alternative. When the suboptimal alternative provided differential signals, the subjects in the 90% FE group developed strong preference for the suboptimal alternative and subjects in the 10% FE group maintained preference for the optimal alternative. The results of both experiments demonstrate that FE trials can have substantial effects on the development of preference in the suboptimal choice task.

Good news is better than bad news, but bad news is not worse than no news.

Under certain conditions, pigeons will reliably prefer an alternative that leads to a lower probability of food over an alternative that leads to a higher probability of food (i.e., demonstrate suboptimal choice). A critical aspect of the typical procedure is that the alternative associated with less food provides differential stimuli that signal trial outcomes, but the alternative associated with more food does not. Few studies have investigated how partial signaling of an alternative influences preference. In Experiments 1-3, pigeons chose between two alternatives that each led to food 60% of the time with partially signaled trial outcomes. One alternative occasionally provided a stimulus that always preceded food (i.e., "good news") and the other alternative occasionally provided a stimulus that always preceded no food ("bad news"). Experiments 2 and 3 also assessed preference in conditions in which alternatives were either completely unsignaled (provided no differential stimuli) or always led to food. Pigeons consistently preferred the "good news" alternative over the "bad news" alternative and preferred 100% food over the "bad news" alternative. The results from conditions in which pigeons chose between the "bad news" alternative and an unsignaled alternative were inconclusive, but suggestive of a preference for bad news. The results are used to evaluate and distinguish between competing explanations of suboptimal choice.

Suboptimal choice and initial-link requirement.

Pigeons (n = 14) were trained in a concurrent-chains suboptimal choice procedure that tested the effect of an increased ratio requirement in the initial links. Fixed-ratio 1 and 25 conditions were manipulated within subjects in a counterbalanced order. In all conditions, distinct terminal-link stimuli on a suboptimal alternative signaled either primary reinforcement (20% of the time) or extinction (80% of the time). On an optimal alternative, two distinct terminal-link stimuli each signaled a 50% chance of primary reinforcement. Preference for the suboptimal alternative was significantly attenuated, and in some birds completely reversed, by the larger response requirement irrespective of condition order. This larger response requirement also generated a notable increase in between-subject variability. A measure of cumulative choice responding is introduced to mitigate the problems associated with traditional session averages. Ordinal predictions of some current theories of suboptimal choice are also considered in light of the results.

Human and pigeon suboptimal choice.

Many studies have shown that pigeons will sometimes behave suboptimally by choosing an option that provides food less frequently over one that provides food more frequently. The critical factor in driving suboptimal behavior in these procedures is that the delayed outcomes are differentially signaled on the suboptimal alternative, but not the optimal alternative. Although this procedure is frequently cited as potentially analogous to human gambling, there is little empirical data to evaluate this assertion. The present study tested both pigeon (Experiment 1) and human (Experiment 2) subjects with a suboptimal choice task. Subjects chose between a suboptimal alternative that provided a large reinforcer 20% of the time and an optimal alternative that always provided a small reinforcer. Stimuli presented during the delays signaled the outcomes on the suboptimal alternative in some conditions. When outcomes were signaled, pigeons chose the suboptimal alternative more frequently than did humans. When the outcomes were not signaled, pigeons' choices became more optimal, but humans' choices did not. Humans' suboptimal choice was unrelated to performance on a probability discounting task. Overall, these findings suggest that although both pigeons and humans can choose suboptimally, more research is needed in order to determine whether non-human performance on this task can serve as a model for human gambling.

Frequency and value both matter in the suboptimal choice procedure.

Pigeons chose between two options on a concurrent-chains task with a single response requirement in the initial link. The suboptimal option ended with food 20% of the time whereas the optimal option ended with food 80% of the time. During a Sig-Both condition, terminal-link stimuli on both options signaled whether or not food would occur. During a Sig-Sub condition, terminal-link stimuli on the suboptimal option provided differential signals, but stimuli on the optimal option did not differentially signal the food and no food outcomes. Initial-link choices revealed a clear preference for the optimal option in the Sig-Both condition, but preference shifted toward suboptimality in the Sig-Sub condition. These findings show that pigeon suboptimal choice is not singularly driven by signal value, as has been suggested, but also by reinforcer frequency.

The influence of outcome delay on suboptimal choice.

Under certain conditions pigeons will choose an option that provides less probable food over one that provides more probable food. This suboptimal choice behavior occurs when the outcomes are delayed and stimuli during the delay differentially signal the upcoming outcomes on the suboptimal alternative, but not the optimal alternative. The present study assessed whether duration of the outcome delay affects pigeons' suboptimal preference. Pigeons chose between a suboptimal alternative that provided food 20% of the time and an optimal alternative that provided food 80% of the time. Stimuli presented during the delays signaled the outcomes on the suboptimal alternative, but not on the optimal alternative. The outcome delays were 5 s in some conditions and 20 s in others. The results of two experiments demonstrate that behavior is generally more suboptimal when the outcome delays are longer but tends to stay relatively suboptimal if subjects experience the long delay condition before the short delay condition. The finding that behavior is more suboptimal with longer delays to the outcomes is consistent with the view that pigeons' suboptimal choice is influenced by both conditioned and primary reinforcement and is inconsistent with the view that suboptimal choice is influenced solely by signal value.

When good news leads to bad choices.

Pigeons and other animals sometimes deviate from optimal choice behavior when given informative signals for delayed outcomes. For example, when pigeons are given a choice between an alternative that always leads to food after a delay and an alternative that leads to food only half of the time after a delay, preference changes dramatically depending on whether the stimuli during the delays are correlated with (signal) the outcomes or not. With signaled outcomes, pigeons show a much greater preference for the suboptimal alternative than with unsignaled outcomes. Key variables and research findings related to this phenomenon are reviewed, including the effects of durations of the choice and delay periods, probability of reinforcement, and gaps in the signal. We interpret the available evidence as reflecting a preference induced by signals for good news in a context of uncertainty. Other explanations are briefly summarized and compared.

When good pigeons make bad decisions: Choice with probabilistic delays and outcomes.

Pigeons chose between an (optimal) alternative that sometimes provided food after a 10-s delay and other times after a 40-s delay and another (suboptimal) alternative that sometimes provided food after 10 s but other times no food after 40 s. When outcomes were not signaled during the delays, pigeons strongly preferred the optimal alternative. When outcomes were signaled, choices of the suboptimal alternative increased and most pigeons preferred the alternative that provided no food after the long delay despite the cost in terms of obtained food. The pattern of results was similar whether the short delays occurred on 25% or 50% of the trials. Shortening the 40-s delay to food sharply reduced suboptimal choices, but shortening the delay to no food had little effect. The results suggest that a signaled delay to no food does not punish responding in probabilistic choice procedures. The findings are discussed in terms of conditioned reinforcement by signals for good news.

Effects of changeover delay on response allocation during probe tests.

The present study assessed whether a pattern of responding that develops when choosing between two alternatives generalizes to novel choice tests when alternatives are presented in new combinations. Pigeons were trained on a two-component multiple schedule. In both components, a concurrent variable-interval (VI) 40-s VI 80-s schedule was used. The COD was 1 s in one component and 10 s in the other. The long COD produced consistently longer dwell times than the short COD did. Following training, subjects were presented with four types of probe-test components in which one alternative was drawn from the component with the short COD and one alternative was drawn from the component with the long COD. When the schedule values of the two alternatives were identical (VI 40 vs. VI 40 and VI 80 vs. VI 80), subjects preferred the alternative trained with the long COD (Ms = .78 and .61, respectively). Additionally, subjects preferred the VI 40-s alternative trained with the long COD to the VI 80-s alternative that was trained with the short COD (M = .85). Systematic preference was not observed when subjects were given a choice between the VI 40-s alternative that was trained with the short COD and the VI 80-s alternative that was trained with the long COD. These results demonstrate that a stimulus associated with a longer COD, and thus longer dwell times in baseline training, may be more preferred during probe tests than expected on the basis of the rate of primary reinforcement associated with that stimulus.

Dual effects on choice of conditioned reinforcement frequency and conditioned reinforcement value.

Pigeons were presented with a concurrent-chains schedule in which the total time to primary reinforcement was equated for the two alternatives (VI 30 s VI 60 s vs. VI 60 s VI 30 s). In one set of conditions, the terminal links were signaled by the same stimulus, and in another set of conditions they were signaled by different stimuli. Choice was in favor of the shorter terminal link when the terminal links were differentially signaled but in favor of the shorter initial link (and longer terminal link) when the terminal links shared the same stimulus. Preference reversed regularly with reversals of the stimulus condition and was unrelated to the discrimination between the two terminal links during the nondifferential stimulus condition. The present results suggest that the relative value of the terminal-link stimuli and the relative rate of conditioned reinforcer presentation are important influences on choice behavior, and that models of conditioned reinforcement need to include both factors.

Discrete-trial vs. continuous free-operant procedures in assessing whether reinforcement context affects reinforcement value.

Pigeons were trained to respond to two alternating concurrent reinforcement schedules. The reinforcement probabilities were .05 and .10 in one component, and .10 and .20 in the other. In one condition, the pigeons received training on a discrete-trial procedure in which the keylights remained illuminated for 5s or until a response occurred. In another condition, pigeons received training on a procedure in which the reinforcement contingencies were the same as in the discrete-trial procedure, but the stimuli were not turned off after 5s or after a response. Following training in each condition, probe tests were presented. In both conditions, the .20 alternative was, overall, preferred to the .05 alternative during probe tests. Following discrete-trial training, there was no reliable preference between the two .10 alternatives. However, when the stimuli remained illuminated during the intertrial interval periods during training, probe tests results showed preference for the .10 alternative that had been presented in the leaner context during training. The pattern of results is consistent with the notion that probe preference can be influenced both by the absolute reinforcement schedules associated with each alternative, as well as changeover behavior developed during training.

The influence of intertrial interval food on extinction and devaluation in chain schedules.

The authors exposed pigeons to 2 equal 3-link chains by using variable-interval schedules of reinforcement. An intertrial interval (ITI) bisected by free food separated the chains. After baseline training, the authors presented terminal links in a successive discrimination to devalue 1 terminal link: The authors reinforced responses to 1 terminal link and extinguished responses to the other. The authors then presented full chains in extinction, except that they continued to deliver free food during the midpoint of the ITI. There were 2 principal findings. First, across all extinction conditions, responding decreased but did not extinguish. Second, when extinction testing revealed a terminal link devaluation effect in the 3rd condition, responding to the initial link was affected, but not middle-link responding. Overall, the results suggested that ITI food presentations can exert a substantial effect on responding in 3-link chain schedules, and they appear to influence both the pattern of extinction and devaluation effects.

Arousal, changeover responses, and preference in concurrent schedules.

Pigeons were trained on multiple schedules that provided concurrent reinforcement in each of two components. In Experiment 1, one component consisted of a variable-interval (VI) 40-s schedule presented with a VI 20-s schedule, and the other a VI 40-s schedule presented with a VI 80-s schedule. After extended training, probe tests measured preference between the stimuli associated with the two 40-s schedules. Probe tests replicated the results of Belke (1992) that showed preference for the 40-s schedule that had been paired with the 80-s schedule. In a second condition, the overall reinforcer rate provided by the two components was equated by adding a signaled VI schedule to the component with the lower reinforcer rate. Probe results were unchanged. In Experiment 2, pigeons were trained on alternating concurrent VI 30-s VI 60-s schedules. One schedule provided 2-s access to food and the other provided 6-s access. The larger reinforcer magnitude produced higher response rates and was preferred on probe trials. Rate of changeover responding, however, did not differ as a function of reinforcer magnitude. The present results demonstrate that preference on probe trials is not a simple reflection of the pattern of changeover behavior established during training.

Inhibition and superconditioning.

Superconditioning is said to occur when learning an association between a conditioned stimulus (CS) and unconditioned stimulus (US) isfacilitated by pairing the CS with the US in the presence of a previously established conditioned inhibitor. Previous demonstrations of superconditioning have been criticized because their control conditions have allowed alternative interpretations. Using a within-subjects autoshaping procedure, the present study unambiguously demonstrated superconditioning. The results support the view that super-conditioning is the symmetric opposite of blocking.