Human Choice Predicted by Obtained Reinforcers, Not by Reinforcement Predictors.

Macphail (1985) proposed that "intelligence" should not vary across vertebrate species when contextual variables are accounted for. Focusing on research involving choice behavior, the propensity for choosing an option that produces stimuli that predict the presence or absence of reinforcement but that also results in less food over time can be examined. This choice preference has been found multiple times in pigeons (Stagner and Zentall, 2010; Zentall and Stagner, 2011; Laude et al., 2014) and has been likened to gambling behavior demonstrated by humans (Zentall, 2014, 2016). The present experiments used a similarly structured task to examine adult human preferences for reinforcement predictors and compared findings to choice behavior demonstrated by children (Lalli et al., 2000), monkeys (Smith et al., 2017; Smith and Beran, 2020), dogs (Jackson et al., 2020), rats (Chow et al., 2017; Cunningham and Shahan, 2019; Jackson et al., 2020), and pigeons (Roper and Zentall, 1999; Stagner and Zentall, 2010). In Experiment 1, adult human participants showed no preference for reinforcement predictors. Results from Experiment 2 suggest that not only were reinforcement predictors not preferred, but that perhaps reinforcement predictors had no effect at all on choice behavior. Results from Experiments 1 and 2 were further assessed using a generalized matching equation, the findings from which support that adult human choice behavior in the present research was largely determined by reinforcement history. Overall, the present results obtained from human adult participants are different than those found from pigeons in particular, suggesting that further examination of Macphail (1985) hypothesis is warranted.

Do pigeons prefer alternatives that include near-hit outcomes?

Pigeons show suboptimal choice on a gambling-like task similar to that shown by humans. Humans also show a preference for gambles in which there are near hits (losses that come close to winning). In the present research, we asked if pigeons would show a preference for alternatives with near-hit-like trials. In Experiment 1, we included an alternative that presented a near hit, in which a stimulus associated with reinforcement (a presumed conditioned reinforcer) changed to a stimulus associated with the absence of reinforcement (a presumed conditioned inhibitor). The pigeons tended to avoid this alternative. In Experiment 2, we varied the duration of the presumed conditioned reinforcer (2 vs. 8 s) that changed to a presumed conditioned inhibitor (8 vs. 2 s) and found that the longer the conditioned reinforcer was presented, the more the pigeons avoided it. In Experiment 3, the near-hit alternative involved an ambiguous stimulus for 8 s that changed to a presumed conditioned reinforcer (or a presumed conditioned inhibitor) for 2 s, but the pigeons still avoided it. In Experiment 4, we controlled for the duration of the conditioned reinforcer by presenting it first for 2 s followed by the ambiguous stimulus for 8 s. Once again, the pigeons avoided the alternative with the near-hit trials. In all 4 experiments, the pigeons tended to avoid alternatives that provided near-hit-like trials. We concluded that humans may be attracted to near-hit trials because near-hit trials give them the illusion of control, whereas this does not appear to be a factor for pigeons.

Further investigation of the Monty Hall Dilemma in pigeons and rats.

In the Monty Hall Dilemma (MHD), three doors are presented with a prize behind one and participants are instructed to choose a door. One of the unchosen doors not containing the prize is revealed, following which the participant can choose to stay with their chosen door or switch to the other one. The optimal strategy is to switch. Herbranson and Schroeder (2010) found that humans performed poorly on this task, whereas pigeons learned to switch readily. We found that pigeons performed only slightly better than humans and that pigeons stayed nearly exclusively when staying and switching were reinforced equally and when staying was the optimal strategy (Stagner et al., 2013b). In Experiment 1 of the present research, rats were trained under these same conditions to observe if possible differences in foraging strategy would influence performance on this task. In Experiment 2, pigeons were trained in an analogous procedure to better compare the two species. We found that both species were sensitive to the overall probability of reinforcement, as both switched significantly more often than subjects that were reinforced equally for staying and switching or reinforced more often for staying. Overall, the two species performed very similarly within the parameters of the current procedure. "This article is part of a Special Issue entitled: Tribute to Tom Zentall."

Suboptimal choice by pigeons may result from the diminishing effect of nonreinforcement.

Pigeons prefer an alternative that provides them with a stimulus 20% of the time that predicts 10 pellets of food and a different stimulus 80% of the time that predicts 0 pellets, over an alternative that provides them with a stimulus that always predicts 3 pellets of food, even though the preferred alternative provides them with considerably less food. It appears that the stimulus that predicts 10 pellets acts as a strong conditioned reinforcer, despite the fact that the stimulus that predicts 0 pellets occurs 4 times as often. In the present research, we tested the hypothesis that early in training conditioned inhibition develops to the 0-pellet stimulus, but later in training it dissipates. We trained pigeons with a hue as the 10-pellet stimulus and a vertical line as the 0-pellet stimulus. To assess the inhibitory value of the vertical line, we compared responding to the 10-pellet hue with responding to the compound of the 10-pellet hue and the vertical line early in training and once again late in training, using both a within-subject design (Experiment 1) and a between-groups design (Experiment 2). We found that there was a significant reduction in inhibition between the early test (when pigeons chose optimally) and late test (when choice was suboptimal). Thus, the increase in suboptimal choice may result from the decline in inhibition to the 0-pellet stimulus. Implications for human gambling behavior are considered.

Midsession reversals with pigeons: visual versus spatial discriminations and the intertrial interval.

Discrimination reversal learning has been used as a measure of species flexibility in dealing with changes in reinforcement contingency. In the simultaneous-discrimination, midsession-reversal task, one stimulus (S1) is correct for the first half of the session, and the other stimulus (S2) is correct for the second half. After training, pigeons show a curious pattern of choices: They begin to respond to S2 well before the reversal point (i.e., they make anticipatory errors), and they continue to respond to S1 well after the reversal (i.e., they make perseverative errors). That is, pigeons appear to be using the passage of time or the number of trials into the session as a cue to reverse, and are less sensitive to the feedback at the point of reversal. To determine whether the nature of the discrimination or a failure of memory for the stimulus chosen on the preceding trial contributed to the pigeons' less-than-optimal performance, we manipulated the nature of the discrimination (spatial or visual) and the duration of the intertrial interval (5.0 or 1.5 s), in order to determine the conditions under which pigeons would show efficient reversal learning. The major finding was that only when the discrimination was spatial and the intertrial interval was short did the pigeons perform optimally.

The Monty Hall dilemma in pigeons: effect of investment in initial choice.

In the Monty Hall dilemma, humans are initially given a choice among three alternatives, one of which has a hidden prize. After choosing, but before it is revealed whether they have won the prize, they are shown that one of the remaining alternatives does not have the prize. They are then asked whether they want to stay with their original choice or switch to the remaining alternative. Although switching results in obtaining the prize two thirds of the time, humans consistently fail to adopt the optimal strategy of switching even after considerable training. Interestingly, there is evidence that pigeons show more optimal switching performance with this task than humans. Because humans often view even random choices already made as being more valuable than choices not made, we reasoned that if pigeons made a greater investment, it might produce an endowment or ownership effect resulting in more human-like suboptimal performance. On the other hand, the greater investment in the initial choice by the pigeons might facilitate switching behavior by helping them to better discriminate their staying versus switching behavior. In Experiment 1, we examined the effect of requiring pigeons to make a greater investment in their initial choice (20 pecks rather than the usual 1 peck). We found that the increased response requirement facilitated acquisition of the switching response. In Experiment 2, we showed that facilitation of switching due to the increased response requirement did not result from extinction of responding to the initially chosen location.

Midsession reversal learning: why do pigeons anticipate and perseverate?

Past research has shown that when given a simultaneous visual-discrimination midsession reversal task, pigeons typically anticipate the reversal well before it occurs and perseverate after it occurs. It appears that they use the estimation of time (or trial number) into the session, rather than (or in addition to) the more reliable cue, the outcome from the previous trial (i.e., a win-stay/lose-shift response rule), to determine which stimulus they should choose. In the present research, we investigated several variables that we thought might encourage pigeons to use a more efficient response strategy. In Experiment 1, we used a treadle-stepping response, rather than key pecking, to test the hypothesis that reflexive key pecking may have biased pigeons to estimate the time (or trial number) into the session at which the reversal would occur. In Experiment 2, we attempted to make the point of reversal in the session more salient by inserting irrelevant trials with stimuli different from the original discriminative stimuli, and for a separate group, we added a 5-s time-out penalty following incorrect choices. The use of a treadle-stepping response did not improve reversal performance, and although we found some improvement in reversal performance when the reversal was signaled and when errors resulted in a time-out, we found little evidence for performance that approached the win-stay/lose-shift accuracy shown by rats.

Guilt by association and honor by association: the role of acquired equivalence.

Guilt by association and honor by association are two types of judgments that suggest that a negative or positive quality of a person or object can transfer to another person or object, merely by co-occurrence. Most examples have been demonstrated under conditions of direct associations. Here, we provide experimental evidence of guilt by association and honor by association via indirect associations. We show that participants may treat two individuals alike if they have been separately paired with a common event using an acquired-equivalence paradigm. Our findings suggest that association fallacies can be examined using a paradigm originally developed for research with nonhuman animals and based on a representation mediation account.

Reversal learning in rats (Rattus norvegicus) and pigeons (Columba livia): qualitative differences in behavioral flexibility.

Research has shown that pigeons given a simultaneous visually based discrimination reversal, in which a single reversal occurs at the midpoint of each session, consistently show anticipation prior to the reversal as well as perseveration after the reversal, suggesting that they use a less effective cue (time or trial number into the session) than what would be optimal to maximize reinforcement (local feedback from the most recent trials). In the present research, pigeons (Columba livia) and rats (Rattus norvegicus) were tested with a simultaneous spatial discrimination midsession reversal. Pigeons showed remarkably similar errors in anticipation and perseveration as with visual stimuli, thereby continuing to show the suboptimal use of time as a cue, whereas rats showed no anticipatory errors and very few perseverative errors, suggesting that they used local feedback as a cue, thus more nearly optimizing reinforcement. To further test the rats' flexibility, they were then tested with a variable point of reversal and then with multiple points of reversal within a session. Results showed that the rats effectively maximized reinforcement by developing an approximation to a win-stay/lose-shift rule. The greater efficiency shown by rats with this task suggests that they are better able to use the feedback from their preceding choice as the basis of their future choice. The difference in cue preference further suggests a qualitative difference in acquisition of the midsession reversal task between pigeons and rats.

Pigeons prefer discriminative stimuli independently of the overall probability of reinforcement and of the number of presentations of the conditioned reinforcer.

When pigeons are given a choice between two alternatives, one leading to a stimulus 20% of the time that always signals reinforcement (S+) or another stimulus 80% of the time that signals the absence of reinforcement (S-) and the other alternative leading to one of two stimuli each signaling reinforcement 50% of the time, the 20% reinforcement alternative is preferred although it provides only 40% as much reinforcement. In Phase 1 of the present experiment, we tested the hypothesis that pigeons compare the S+ associated with each alternative and ignore the S- by giving them a choice between two pairs of discriminative stimuli (20% S+, 80% S- and 50% S+, 50% S-). Reinforcement theory suggests that the alternative associated with more reinforcement should be preferred but the pigeons showed indifference. In Phase 2, the pigeons were divided into two groups. For one group, the discriminative function was removed from the 50% reinforcement alternative and a strong preference for the 20% reinforcement alternative was found. For the other group, the discriminative function was removed from both alternatives and a strong preference was found for the 50% reinforcement alternative. Thus, the indifference found in Phase 1 was not due to the absence of discriminability of the differential reinforcement associated with the two alternatives (20% vs. 50% reinforcement); rather, the indifference can be attributed to the pigeons' insensitivity to the differential frequency of the two S+ and two S- stimuli. The relevance to human gambling behavior is discussed.

Do pigeons prefer information in the absence of differential reinforcement?

Prior research has indicated that pigeons do not prefer an alternative that provides a sample (for matching to sample) over an alternative that does not provide a sample (i.e., there is no indication of which comparison stimulus is correct). However, Zentall and Stagner (Journal of Experimental Psychology. Animal Behavior Processes 36:506-509, 2010) showed that when delay of reinforcement was controlled, pigeons had a strong preference for matching over pseudomatching (i.e., there was a sample, but it did not indicate which comparison stimulus was correct). Experiment 1 of the present study replicated and extended the results of the Zentall and Stagner (Journal of Experimental Psychology. Animal Behavior Processes 36:506-509, 2010) study by including an identity relation between the sample and one of the comparison stimuli in both the matching and pseudomatching tasks. In Experiment 2, in which we asked whether the pigeons would still prefer matching if we equated the two tasks for probability of reinforcement, we found no systematic preference for matching over pseudomatching. Thus, it appears that in the absence of differential reinforcement, the information provided by a sample that signals which of the two comparison stimuli is correct is insufficient to produce a preference for that alternative.

Sub-Optimal Choice by Pigeons: Failure to Support The Allais Paradox.

Pigeons show a preference for an alternative that provides them with discriminative stimuli (sometimes a stimulus that predicts reinforcement and at other times a stimulus that predicts the absence of reinforcement) over an alternative that provides them with non discriminative stimuli, even if the non discriminative stimulus alternative is associated with 2.5 times as much reinforcement (Stagner & Zentall, 1910). In Experiment 1 we found that the delay to reinforcement associated with the non discriminative stimuli could be reduced by almost one half before the pigeons were indifferent between the two alternatives. In Experiment 2 we tested the hypothesis that the preference for the discriminative stimulus alternative resulted from the fact that, like humans, the pigeons were attracted by the stimulus that consistently predicted reinforcement (the Allais paradox). When the probability of reinforcement associated with the discriminative stimulus that predicted reinforcement was reduced from 100% to 80% the pigeons still showed a strong preference for the discriminative stimulus alternative. Thus, under these conditions, the Allais paradox cannot account for the sub-optimal choice behavior shown by pigeons. Instead we propose that sub-optimal choice results from positive contrast between the low expectation of reinforcement associated with the discriminative stimulus alternative and the much higher obtained reinforcement when the stimulus associated with reinforcement appears. We propose that similar processes can account for sub-optimal gambling behavior by humans.

Pigeons prefer conditional stimuli over their absence: a comment on Roberts et al. (2009).

Recently, Roberts et al. (2009) have suggested that pigeons performing delayed matching-to-sample appear unwilling to request to see the sample again (or even for the first time) prior to choice, even if that choice would result in an increase in matching accuracy. In each of their four experiments, however, presentation (Experiments 3 & 4) or representation of the sample (Experiments 1 & 2) resulted in an added delay to reinforcement. Thus, the pigeons had to choose between an immediate reinforcer on about 50% of the trials and a delayed reinforcer on a significantly higher percentage of the trials. In the present research, when we equated the two alternatives for delay to reinforcement, we found that pigeons generally showed a significant preference for trials with a relevant sample over trials with an irrelevant sample. When the contingencies were reversed, most of the pigeons reversed their preference. Although these results do not present evidence for metacognition, they do show that pigeons are sensitive to the potential for a higher probability of reinforcement when delay to reinforcement is controlled.

Suboptimal choice behavior by pigeons.

Contrary to the law of effect and optimal foraging theory, pigeons show suboptimal choice behavior by choosing an alternative that provides 20% reinforcement over another that provides 50% reinforcement. They choose the 20% reinforcement alternative--in which 20% of the time, that choice results in a stimulus that always predicts reinforcement, and 80% of the time, it results in another stimulus that predicts its absence--rather than the 50% reinforcement alternative, which results in one of two stimuli, each of which predicts reinforcement 50% of the time. This choice behavior may be related to suboptimal human monetary gambling behavior, because in both cases, the organism overemphasizes the infrequent occurrence of the winning event and underemphasizes the more frequent occurrence of the losing event.

Episodic-like memory: pigeons can report location pecked when unexpectedly asked.

Pigeons were tested for their ability to report the location they recently pecked, without prior experience having to do so. They were first pretrained to report the location that they had just pecked. They were then trained on a conditional discrimination to associate yellow and blue samples with vertical and horizontal comparisons, respectively, independent of comparison location. On probe trials in testing, when after choosing a vertical or horizontal line following the yellow or blue sample, the pigeons were 'asked' which location they had just pecked, they showed a significant tendency to choose correctly in spite of the fact that location of the correct comparison was incidental to the task. Performing on probe trials is analogous to asking the pigeons an unexpected question about their recent behavior and it is similar to the episodic memory question asked of humans, "What did you have for breakfast this morning?".