Stimuli inevitably generated by behavior that avoids electric shock are inherently reinforcing.

A molecular analysis based on the termination of stimuli that are positively correlated with shock and the production of stimuli that are negatively correlated with shock provides a parsimonious count for both traditional discrete-trial avoidance behavior and the data derived from more recent free-operant procedures. The necessary stimuli are provided by the intrinsic feedback generated by the subject's behavior, in addition to those presented by the experimenter. Moreover, all data compatible with the molar principle of shock-frequency reduction as reinforcement are also compatible with a delay-of-shock gradient, but some data compatible with the delay gradient are not compatible with frequency reduction. The delay gradient corresponds to functions relating magnitude of behavioral effect to the time between conditional and unconditional stimuli, the time between conditioned and primary reinforcers, and the time between responses and positive reinforcers.

Stimulus control: part I.

In his effort to distinguish operant from respondent conditioning, Skinner stressed the lack of an eliciting stimulus and rejected the prevailing stereotype of Pavlovian "stimulus-response" psychology. But control by antecedent stimuli, whether classified as conditional or discriminative, is ubiquitous in the natural setting. With both respondent and operant behavior, symmetrical gradients of generalization along unrelated dimensions may be obtained following differential reinforcement in the presence and the absence of the stimulus. The slopes of these gradients serve as measures of stimulus control, and they can be steepened without applying differential reinforcement to any two points along the test dimension. Increases and decreases in stimulus control occur under the same conditions as those leading to increases and decreases in observing responses, indicating that it is the increasing frequency and duration of observation (and perhaps also of attention) that produces the separation in performances during discrimination learning.

Stimulus control: part II.

The second part of my tutorial stresses the systematic importance of two parameters of discrimination training: (a) the magnitude of the physical difference between the positive and the negative stimulus (disparity) and (b) the magnitude of the difference between the positive stimulus, in particular, and the background stimulation (salience). It then examines the role these variables play in such complex phenomena as blocking and overshadowing, progressive discrimination training, and the transfer of control by fading. It concludes by considering concept formation and imitation, which are important forms of application, and recent work on equivalence relations.

The autoshaping procedure as a residual block clock.

In the first experiment, 4 pigeons were each presented with a recurring sequence of four key colors followed by the delivery of grain (block clock). Once the rate of pecking had stabilized, three of the colors were replaced, during different series of sessions, by a darkening of the key. The rate of pecking was reduced within those segments of the interval between deliveries of food during which the key was dark; when the key was dark during the final portion of the interval, rates were reduced throughout the entire interval. In the second experiment, 3 new pigeons were exposed to a different sequence of colors, and the final stimulus was replaced in successive conditions by a novel color, a darkened key, and a restoration of the original color. The data indicated that darkening the key had a more severe, more extensive, and more persistent effect than did a mere change in color. These results suggest that it may be fruitful to conceptualize the autoshaping procedure as a special version of the block clock in which pecking is suppressed throughout the greater part of the interval by darkening the key. In the final condition, the same stimulus appeared in each of the last three portions of the interval. The rate of pecking was lower during the last two portions than when distinctive colors were presented, with the peak rate now appearing in the fifth of seven equal temporal components.

Information on response requirements compared with information on food density as a reinforcer of observing in pigeons.

On a variable-interval schedule, pecking the key to the pigeon's right (observing response) produced red or green displays relating to the delivery of grain and its dependence on pecking the key to the left (food key). During various blocks of sessions, mixed (no stimulus change) schedules including the following pairs of components were temporarily converted by the observing response to their corresponding multiple (correlated stimuli) schedules: variable-interval 60-s, extinction; variable-interval 60-s, variable-time (response-independent) 60-s; extinction, variable-time 60-s. Differences in food delivery maintained substantial rates of responding on the observing key, without regard to pecking requirements on the food key. Although stimuli correlated with differences in the response requirement on the food key maintained higher observing rates than those maintained by uncorrelated stimuli, they were much lower than those based on food. The value of predictive stimuli as reinforcers is determined by the value of the events predicted. In particular, the cost of pecking appears to be low, and this may place limitations on the applicability of energy-based and economic models of behavior.

Escape from serial stimuli leading to food.

If the functional relations governing the strength of a conditioned reinforcer correspond to those obtained with other Pavlovian procedures (e.g., Kaplan, 1984), the termination of stimuli appearing early in the interval between successive food deliveries should be reinforcing. During initial training we presented four key colors, followed by food, in a recurrent sequence to each of 6 pigeons. This established a baseline level of autoshaped pecking. In later sessions, we terminated each of these colors or only the first color for a brief period following each peck, replacing the original color with a standard substitute to avoid darkening the key. Pecking decreased in the presence of the last color in the sequence but increased in the presence of the first. In accord with contemporary models of Pavlovian conditioning, these and other data suggest that the behavioral effects of stimuli in a chain may be better understood in terms of what each stimulus predicts, as measured by relative time to the terminal reinforcer, than in the exclusively positive terms of the traditional formulation (Skinner, 1938). The same model may also account for the initial pause under fixed-interval and fixed-ratio schedules of reinforcement.

The effect of negative stimulus presentations on observing-response rates.

Theories of observing differ in predicting whether or not a signal for absence of reinforcement (S-) is capable of reinforcing observing responses. Experiments in which S- was first removed from and then restored to the procedure have yielded mixed results. The present experiments suggest that failure to control for the direct effect of presenting S- may have been responsible. Pigeons and operant procedures were used. Experiment 1 showed that presentations of S-, even when not contingent on observing, can raise the rate of an observing response that was reinforced only by presentations of a signal (S+) that accompanied a schedule of food delivery. Experiment 2 showed that this effect resulted from bursts of responding that followed offsets of S-. Experiment 3 showed that, when the presence of S- was held constant, lower rates occurred when S- was dependent on, rather than independent of, observing. These results support theories that characterize S- as incapable of reinforcing observing responses.

The role of observing and attention in establishing stimulus control.

Early theorists (Skinner, Spence) interpreted discrimination learning in terms of the strengthening of the response to one stimulus and its weakening to the other. But this analysis does not account for the increasing independence of the two performances as training continues or for increases in control by dimensions of a stimulus other than the one used in training. Correlation of stimuli with different densities of reinforcement produces an increase in the behavior necessary to observe them, and greater observing of and attending to the relevant stimuli may account for the increase in control by these stimuli. The observing analysis also encompasses errorless training, and the selective nature of observing explains the feature-positive effect and the relatively shallow gradients of generalization generated by negative discriminative stimuli. The effectiveness of the observing analysis in handling these special cases adds to the converging lines of evidence supporting its integrative power and thus its validity.

Testing the reinforcing properties of S-: a replication of Lieberman's procedure.

A critical issue in testing theories of observing is whether the stimulus associated with extinction (the S-) reinforces observing responses. In previous experiments, subjects have been trained to make observing responses that produce both the S- and the stimulus correlated with reinforcement (the S+). Then, either the S+ or the S- has been withheld. Conflicting results have been attributed to differences among species. In the present experiments, pecking one key by master pigeons was reinforced with grain on a variable-ratio extinction schedule. Yoked pigeons received the grain on a variable-interval, extinction schedule controlled by the variable-ratio performances of the master birds. For both groups, concurrent pecking on a second key was reinforced on a variable-interval schedule with displays of discriminative stimuli. Subsequently, either the S+ or the S- was eliminated from the procedure. Omission of S+ produced a large decrease, as predicted by traditional conditioned reinforcement accounts of observing. By itself, S- did not maintain observing. A smaller and less reliable decrease, comparable to that obtained by Lieberman (1972) with rhesus monkeys, occurred when S- was eliminated. This replication with pigeons of Lieberman's results indicates that they are not species-specific, and the fact that observing was not maintained by S- alone suggests that the decrease obtained when S- was omitted is not attributable to the reinforcing power of S-.

Separating the effects of salience and disparity on the rate of observing.

Pigeons producing deliveries of grain on a mixed variable-interval, extinction schedule by pecking a center key could also produce discriminative stimuli on concurrent variable-interval schedules by pecking the left or right observing key. The stimuli produced by each observing key were varied independently. In the first experiment, the negative discriminative stimulus was at the far end of the spectrum from the key illumination accompanying the mixed schedule and from the positive discriminative stimulus. When the magnitude of the difference between the latter two stimuli (salience) was varied, more pecks occurred on the observing key producing the larger of the two differences than on the key producing the smaller difference. In the second experiment, the stimulus accompanying the mixed schedule was at the far end of the spectrum, and the magnitude of the difference between the two discriminative stimuli (disparity) was varied. The proportion of pecks occurring on each observing key shifted systematically in the direction of the key producing the larger difference. The salience of the discriminative stimuli and their disparity each has an independent influence on the frequency of observing when the other is controlled, but the effect of the salience appears to be the more substantial.

Spatial and temporal relations in conditioned reinforcement and observing behavior.

In Experiment 1, depressing one perch produced stimuli indicating which of two keys, if pecked, could produce food (spatial information) and depressing the other perch produced stimuli indicating whether a variable-interval or an extinction schedule was operating (temporal information). The pigeons increased the time they spent depressing the perch that produced the temporal information but did not increase the time they spent depressing the perch that produced the spatial information. In Experiment 2, pigeons that were allowed to produce combined spatial and temporal information did not acquire the perch pressing any faster or maintain it at a higher level than pigeons allowed to produce only temporal information. Later, when perching produced only spatial information, the time spent depressing the perch eventually declined. The results are not those implied by the statement that information concerning biologically important events is reinforcing but are consistent with an interpretation in terms of the acquisition of reinforcing properties by a stimulus associated with a higher density of primary reinforcement.

The acquisition of observing.

Pigeons were exposed to stimuli correlated with the presence or absence of a variable-interval 60-second schedule of reinforcement only while they depressed a crossbar or "perch." In the first experiment, the stimuli were different tilts of a line displayed on the key. When the difference in brightness between the line and the background (salience) was maximal, seven of eight birds acquired the discrimination, but when the difference was reduced by 50%, only one succeeded. In the second experiment, wavelength of chamber illumination served as the relevant dimension. Neither experiment showed a large effect attributable to the magnitude of the difference (disparity) between the positive and the negative stimulus. Individual differences in time spent observing were positively correlated with level of discrimination in the presence of the stimuli. All birds produced the positive stimulus for a greater proportion of the available time than they did the negative stimulus. This may be the mechanism that provides selective reinforcement of observing. Finally, the formation of a discrimination was analyzed in terms of changes in the proportion of time spent in contact with the discriminative stimuli.

Conditioned reinforcement as a function of duration of stimulus.

Pigeons were provided with three keys. Pecking the center key produced grain on a schedule that alternated at unpredictable times between a variable-interval component and extinction. On concurrent variable-interval schedules, pecking either side key produced a stimulus associated with the variable-interval component on the center key provided that said schedule was currently in effect. The independent variable was the length of time this stimulus remained on the keys. Pecking one side key produced the stimulus for 27 seconds, whereas the duration produced by pecking the other key varied for successive blocks of sessions. For the first four birds, the values tested were 3, 9, 27, and 81 seconds. For the second group, numbering three birds, the values tested were 1, 3, 9, and 27 seconds. The dependent variable was the proportion of total side key pecks that occurred on the variable key. For all birds, the function was positive in slope and negative in acceleration. This finding supports a formulation that ascribes the maintenance of observing responses in a normal setting to the fact that the subject exposes itself to the positive discriminative stimulus for a longer mean duration than it does to the negative stimulus.