A theory of behaviour on progressive ratio schedules, with applications in behavioural pharmacology.

RATIONALE: Mathematical principles of reinforcement (MPR) provide the theoretical basis for a family of models of schedule-controlled behaviour. A model of fixed-ratio schedule performance that was applied to behaviour on progressive ratio (PR) schedules showed systematic departures from the data. OBJECTIVE: This study aims to derive a new model from MPR that will account for overall and running response rates in the component ratios of PR schedules, and their decline toward 0, the breakpoint. METHOD: The role of pausing is represented in a real-time model containing four parameters: T (0) and k are the intercept and slope of the linear relation between post-reinforcement pause duration and the prior inter-reinforcer interval; a (specific activation) measures the incentive value of the reinforcer; δ (response time) sets biomechanical limits on response rate. Running rate is predicted to decrease with negative acceleration as ratio size increments, overall rate to increase and then decrease. Differences due to type of progression are explained as hysteresis in the control by reinforcement rates. Re-analysis of extant data focuses on the effects of acute treatment with antipsychotic drugs, lesions of the nucleus accumbens core, and destruction of orexinergic neurones of the lateral hypothalamus. RESULTS: The new model resolves some anomalies evident in earlier analyses, and provides new insights to the results of these interventions. CONCLUSIONS: Because they can render biologically relevant parameters, mathematical models can provide greater power in interpreting the effects of interventions on the processes underlying schedule-controlled behaviour than is possible for first-order data such as the breakpoint.

No thanks, I'm good. Any more and I'll be sick: comment on Lynch and Carroll (2001).

W. J. Lynch and M. E. Carroll's (2001) excellent analyses of drug intake from a regulation perspective are formalized in terms of control systems. Satiation corresponds to the set point, deviations below which are called hunger or craving, deviations above which are called surfeit. Although simple, the model provides a unifying framework for many of the phenomena Lynch and Carroll describe.

The principal components of response strength.

As Skinner (1938) described it, response strength is the "state of the reflex with respect to all its static properties" (p. 15), which include response rate, latency, probability, and persistence. The relations of those measures to one another was analyzed by probabilistically reinforcing, satiating, and extinguishing pigeons' key pecking in a trials paradigm. Reinforcement was scheduled according to variable-interval, variable-ratio, and fixed-interval contingencies. Principal components analysis permitted description in terms of a single latent variable, strength, and this was validated with confirmatory factor analyses. Overall response rate was an excellent predictor of this state variable.

Writing and overwriting short-term memory.

An integrative account of short-term memory is based on data from pigeons trained to report the majority color in a sequence of lights. Performance showed strong recency effects, was invariant over changes in the interstimulus interval, and improved with increases in the intertrial interval. A compound model of binomial variance around geometrically decreasing memory described the data; a logit transformation rendered it isomorphic with other memory models. The model was generalized for variance in the parameters, where it was shown that averaging exponential and power functions from individuals or items with different decay rates generates new functions that are hyperbolic in time and in log time, respectively. The compound model provides a unified treatment of both the accrual and the dissipation of memory and is consistent with data from various experiments, including the choose-short bias in delayed recall, multielement stimuli, and Rubin and Wenzel's (1996) meta-analyses of forgetting.

Modeling games from the 20th century.

A scientific framework is described in which scientists are cast as problem-solvers, and problems as solved when data are mapped to models. This endeavor is limited by finite attentional capacity which keeps depth of understanding complementary to breadth of vision; and which distinguishes the process of science from its products, scientists from scholars. All four aspects of explanation described by Aristotle trigger, function, substrate, and model are required for comprehension. Various modeling languages are described, ranging from set theory to calculus of variations, along with exemplary applications in behavior analysis.

How the propagation of error through stochastic counters affects time discrimination and other psychophysical judgments.

The performance of fallible counters is investigated in the context of pacemaker-counter models of interval timing. Failure to reliably transmit signals from one stage of a counter to the next generates periodicity in mean and variance of counts registered, with means power functions of input and standard deviations approximately proportional to the means (Weber's law). The transition diagrams and matrices of the counter are self-similar: Their eigenvalues have a fractal form and closely approximate Julia sets. The distributions of counts registered and of hitting times approximate Weibull densities, which provide the foundation for a signal-detection model of discrimination. Different schemes for weighting the values of each stage may be established by conditioning. As higher order stages of a cascade come on-line the veridicality of lower order stages degrades, leading to scale-invariance in error. The capacity of a counter is more likely to be limited by fallible transmission between stages than by a paucity of stages. Probabilities of successful transmission between stages of a binary counter around 0.98 yield predictions consistent with performance in temporal discrimination and production and with channel capacities for identification of unidimensional stimuli.

Satiation, capacity, and within-session responding.

Responding may change substantially over the course of a session (McSweeney, Hinson, & Cannon, 1996). The role of satiation in this effect was investigated in three experiments. Experiment 1 showed that the capacity of pigeons to consume milo over a 1-hr period was relatively stable across three different methods of measurement. In Experiment 2, pigeons were divided into two groups that differed in their capacity based on one of those measures. Key pecking was then reinforced under a variable-interval 30-s schedule with hopper durations of 2 or 5 s. According to the satiation hypothesis, subjects with small capacities should satiate faster and therefore show greater decreases in food-reinforced responding than would subjects with larger capacities. The results confirmed this prediction and showed that the magnitudes of within-session decreases were better predicted by the amount an animal consumed relative to its capacity than by absolute amount alone. In Experiment 3, each pigeon was prefed 0, 5, 15, or 25 g of milo prior to each session. Consistent with the satiation hypothesis, increases in prefeeding produced lower overall response rates in the smaller capacity subjects than in the larger capacity subjects at each level of prefeeding. These experiments demonstrate the importance of a new variable in the control of behavior, and provide a recommended technique for its measurement.

Modeling modeling.

Models are tools; they need to fit both the hand and the task. Presence or absence of a feature such as a pacemaker or a cascade is not in itself good. Or bad. Criteria for model evaluation involve benefit-cost ratios, with the numerator a function of the range of phenomena explained, goodness of fit, consistency with other nearby models, and intangibles such as beauty. The denominator is a function of complexity, the number of phenomena that must be ignored, and the effort necessary to incorporate the model into one's parlance. Neither part of the ratio can yet be evaluated for MTS, whose authors provide some cogent challenges to SET.

A clock not wound runs down.

Seven pigeons were trained to respond on one key during the first half of a 60-s trial, and on another during the second half, to test the prediction that the rate of the internal pacemaker should slow in the absence of reward. The relative probability of responding was well-described by Erlang distributions, which yielded as parameters the period of the pacemaker and the criterial number of counts for switching to the second key. During a subsequent period of extinction the period increased linearly with time in extinction, and the criterial count decreased. This slowing of the pacemaker was predicted by the Behavioral Theory of timing, but not by other theories.

Satiation causes within-session decreases in instrumental responding.

The amount of food necessary to cause within-session decreases in response rates was investigated by varying reinforcer duration (Experiment 1) and grain size (Experiment 2). Within-session response rates quickly increases to a peak, and the decrease throughout the rest of the session was greater for conditions with larger amounts of food. The decreases were greatest for the pigeons that ate the least when food was freely available. Previous studies of within-session changes in responding are reviewed and reinterpreted. It was concluded that satiation causes the decrease in responding and that this effect is not restricted to extreme manipulations of the amount of food consumed.

Models of ratio schedule performance.

Predictions of P. R. Killeen's (1994) mathematical principles of reinforcement were tested for responding on ratio reinforcement schedules. The type of response key, the number of sessions per condition, and first vs. second half of a session had negligible effects on responding. Longer reinforcer durations and larger grain types engendered more responding, affecting primarily the parameter alpha (specific activation). Key pecking was faster than treadle pressing, affecting primarily the parameter delta (response time). Longer intertrial intervals led to higher overall response rates and shorter postreinforcement pauses and higher running rates, and ruled out some competing explanations. The treadle data required a distinction between the energetic requirements and rate-limiting properties of extended responses. The theory was extended to predict pause durations and run rates on ratio schedules.

Bayesian analysis of foraging by pigeons (Columba livia).

In this article, the authors combine models of timing and Bayesian revision of information concerning patch quality to predict foraging behavior. Pigeons earned food by pecking on 2 keys (patches) in an experimental chamber. Food was primed for only 1 of the patches on each trial. There was a constant probability of finding food in a primed patch, but it accumulated only while the animals searched there. The optimal strategy was to choose the better patch first and remain for a fixed duration, thereafter alternating evenly between the patches. Pigeons were nonoptimal in 3 ways: (a) they departed too early, (b) their departure times were variable, and (c) they were biased in their choices after initial departure. The authors review various explanations of these data.

Economics, ecologics, and mechanics: The dynamics of responding under conditions of varying motivation.

The mechanics of behavior developed by Killeen (1994) is extended to deal with deprivation and satiation and with recovery of arousal at the beginning of sessions. The extended theory is validated against satiation curves and within-session changes in response rates. Anomalies, such as (a) the positive correlation between magnitude of an incentive and response rates in some contexts and a negative correlation in other contexts and (b) the greater prominence of incentive effects when magnitude is varied within the session rather than between sessions, are explained in terms of the basic interplay of drive and incentive motivation. The models are applied to data from closed economies in which changes of satiation levels play a key role in determining the changes in behavior. Relaxation of various assumptions leads to closed-form models for response rates and demand functions in these contexts, ones that show reasonable accord with the data and reinforce arguments for unit price as a controlling variable. The central role of deprivation level in this treatment distinguishes it from economic models. It is argued that traditional experiments should be redesigned to reveal basic principles, that ecologic experiments should be redesigned to test the applicability of those principles in more natural contexts, and that behavioral economics should consist of the applications of these principles to economic contexts, not the adoption of economic models as alternatives to behavioral analysis.

Categorical scaling of time: implications for clock-counter models.

Pigeons partitioned time into three intervals. Responses to one key could be reinforced after a short time, to a second key after an intermediate time, and to a third key after a long time. The values of the short, intermediate, and long times and the proportion of trials ending with reinforcement were varied. Absolute and relative response rates on each key were an orderly function of time and showed approximately proportional changes with changes in the interval values, consistent with Weber's law, Gibbon's (1977) scalar expectancy theory, and Killeen and Fetterman's (1988) behavioral theory of timing (BeT). Standard deviations of the times at which subjects switched between successive keys increased more slowly within a condition than across conditions, as predicted by BeT. Increases and decreases in reinforcement probability produced both transient and longer lasting changes in timing behavior, once again, in accord with predictions of BeT.

Frustration: Theory and practice.

Frustration is often dismissed as a transient by-product of thwarted aspirations, a disruptive and uncivilized mark of Cain. Amsel's work, however, shows the creative and enabling role that frustration can play in the behavior of organisms. The book epitomized here first clarifies the basic phenomenon and its causes, and then extends it by mapping its development, along with that of other behavioral markers, against the development of brain structures. One may take exception to the particulars: Are the chosen variables the best ones to measure? Is frustration an autonomous motive or is it the liberation of the arousal normally focused on the instrumental response? Is the best reading always given to the large and heterogeneous literature? But the whole of Amsel's work transcends these particulars and exemplifies, as do few other curriculum vitae, the ideal of systematic scientific inquiry that is praised more often than practiced.

Scaling pigeons' choice of feeds: bigger is better.

Preferences of hungry pigeons among 10 grains and pellets were analyzed using a Thurstone scaling procedure. The recovered scales were positively correlated with size of the feed. The correlations improved when the Thurstonian assumption of equal-sized discriminal dispersions (Case V) was replaced with the assumption of proportional-sized dispersions (Case VI), as entailed by Weber's law. The correlations weakened when the experiments were conducted with the pigeons close to their free-feeding weights, where the probability of sampling alternative grains increased. In the final experiment, exposure to a large pellet shifted the preferences between two smaller pellets.

The behavioral theory of timing: transition analyses.

Gibbon and Church (1990, 1992) have recently confirmed an important, parameter-free prediction of the behavioral theory of timing (Killeen & Fetterman, 1988): The times of exiting from a bout of activity are positively correlated with the times of entrance to it. The correlations were slightly less than predicted, however, and the correlations between the start of an activity and the time spent engaged in that activity were negative, rather than zero. We adapted their serial model as an augmented (one-parameter) version of the behavioral theory, positing a lag between the receipt of a pulse from the pacemaker and transition into the next class of responses. The augmented version of the behavioral theory further improved the correspondence between the theory and the correlational data reported by Gibbon and Church. It also accounts for previously unpublished data from our laboratory derived from a new timing technique, the "peak choice" procedure. We show that the measured variance of movement times from one key to another closely approximates the estimated variance of transition times recovered from fits of the augmented model to the data. Such correspondence both attests to the correct identification of this source of variance and suggests ways to remove it, both from behavior and from our models of behavior.

Changing rates of reinforcement perturbs the flow of time.

This experiment tested the hypothesis that changes in rate of reinforcement affect the rate of an internal pacemaker as suggested by the Behavioral Theory of Timing (BeT: Killeen and Fetterman, 1988). Pigeons were trained to discriminate durations of 10 s and 20 s, and then exposed to higher or lower rates of freely delivered reinforcers. When returned to the discrimination task, judgments were reliably biased in the predicted directions: those returning from a richer context judged standard durations to be longer than did those returning from a poorer context. These results validate a key assumption of BeT, and provide an explanation of how changing tempos of life bias the perception of time.

Mechanics of the animate.

Behavior is treated as basic physics. Dimensions are identified and their transformations from physical specification to axes in behavioral space are suggested. Responses are treated as action patterns arrayed along a continuum of activation energy. Behavior is seen as movement along a trajectory through this behavior space. Incentives or reinforcers are attractors in behavior space, at the centers of basins of lowered potential. Trajectories impinging on such basins may be captured; repeated capture will warp the trajectory toward a geodesic, a process called conditioning. Conditioning is enhanced by contiguity, the proximity between the measured behavior and the incentive at the end of the trajectory, and by contingency, the depth of the trajectory below the average level of the potential energy landscape. Motivation is seen as the potential of an organism for motion under the forces impinging on it. Degree of motivation is characterized by the depth of the potential field, with low motivation corresponding to a flat field and a flat gradient of activation energy. Drives are the forces of incentives propagated through behavior space. Different laws for the attenuation of drive with behavioral distance are discussed, as is the dynamics of action. The basic postulate of behavior mechanics is incentive-tracking in behavior space, the energy for which is provided by decreases in potential. The relation of temporal gradients to response differentiation and temporal discrimination is analyzed. Various two-body problems are sketched to illustrate the application of these ideas to association, choice, scalar timing, self-control, and freedom.