Rate matching, probability matching, and optimization in concurrent ratio schedules.

Much research has documented rate matching in concurrent variable-interval schedules, but comparatively little research has examined performance in concurrent variable-ratio schedules, except in discrete-trials procedures that sometimes produce probability matching. One should expect that the two types of schedule would result in different performances, because ratio schedules cannot improve with time the way interval schedules do; ratio schedules lack the temporal dynamics of interval schedules. The present experiment exposed rats to concurrent variable-ratio schedules. Seven unsignaled components were presented in random order within each daily session, with probability ratios ranging from 1:8 to 8:1. Three conditions were studied that varied the overall probability of food while leaving probability ratios the same. Choice appeared to conform to probability matching, because sensitivities in the rate-matching relation were close to 0.5, whereas sensitivities to probability ratio were close to 1.0. The sensitivities alone, however, could not confirm probability matching, because undermatching to rate occurs often. Analyses at smaller time scales supported the interpretation of probability matching. In particular, control by food deliveries was highly local in concurrent variable-ratio schedules, in contrast with concurrent variable-interval schedules, in which control is extended. Activity continued to switch between alternatives throughout components, contradicting optimal sampling theory.

Analyzing the magnitude effect in spontaneously hypertensive (SHR) and wistar Kyoto (WKY) rats.

We analyzed the magnitude effect in Spontaneously Hypertensive (SHR) and Wistar Kyoto (WKY) rats choosing between a smaller-sooner (SSF) and a larger-later food (LLF) in the initial link of a concurrent-chains procedure. The SSF was delivered immediately in one terminal link and the LLF delayed 0.01, 5, 10, 20, 40, or 80 s in the other terminal link. An ABABA design varied food amount, 1 vs. 4 and 3 vs. 8 food-pellets in conditions A and B, respectively. The SHRs made more impulsive choices than the WKYs. The hyperbolic-decay model and the Generalized Matching Law fitted the data well. Discounting rate (k) and the area under the discounting curve (AUC) for the choices made by the SHRs in conditions A, were like those in conditions B. For the choices that the individual WKYs made, k was slightly higher and AUC smaller in conditions B than in conditions A. For both strains sensitivity to the immediacy of the LLF (s) was slightly higher in conditions A than in conditions B. Thus, we found no conclusive, compelling evidence either supporting or discarding the magnitude effect in the SHRs and scarce evidence supporting an effect opposite to the magnitude effect in the WKYs.

Response-reinforcer contiguity versus response-rate-reinforcer-rate covariance in rats' lever pressing: Support for a multiscale view.

The multiscale molar view sees behavior as a flow, like a river, extended in time. Matching theory expresses the way activities compete for time. Relative time taken by any activity depends on relative induction. The present experiment tested matching theory applied to concurrent contingent and noncontingent food. As adjunctive activities that compete with operant activity, we recorded hopper head entries and presses on a lever near the food hopper that had no programmed consequences. Eight naïve rats were first exposed to a variable-time 60 s schedule, which across conditions was gradually transformed into a variable-interval 60 s schedule by increasing the proportion of food that was delivered contingent on pressing a lever far from the hopper. Another group of 4 rats that had been trained to press a lever near a food hopper were introduced in the second condition, in which one food delivery was contingent on far-lever pressing. We found induction following a power function to describe pressing on the far lever (operant activity). Matching theory combined with power-function induction also accounted for adjunctive activity. Results with single contingent food deliveries provided little support for the molecular view that behavior consists of discrete responses "strengthened" by immediately following reinforcers.

Spontaneously hypertensive (SHR) rats choose more impulsively than Wistar-Kyoto (WKY) rats on a delay discounting task.

Indications of Attention Deficit Hyperactivity Disorder (ADHD) in the spontaneously hypertensive rat (SHR) are not consistent across different tests of impulsivity, questioning the SHR's validity as a rodent model of ADHD. This study used a concurrent-chains procedure to examine possible differences in impulsive choice between SHRs and control-normotensive Wistar Kyoto (WKY) rats. The aim was to extend the generality of findings showing regularities between the hyperbolic-decay model and the generalized matching law fitting delay discounting data from nonhuman animals. The objectives were to: (1) examine differences in impulsive choice between SHRs and WKYs; (2) add evidence suggesting that the SHR is a suitable model of ADHD; (3) demonstrate that concurrent-chains procedures requiring locomotion detect differences in impulsive choice between SHRs and WKYs; (4) support the idea that impulsivity in nonhuman animals increases with training. The initial link used two non-independent random interval schedules arranging entries to the terminal links, where one fixed-time (FT) schedule delayed 1-food pellet and the other FT 4-food pellets. The FT delaying the former was kept constant at 0.1s and that delaying the latter changed after every 10 food deliveries, defining six delay components (0.1, 5, 10, 20, 40, and 80s) presented in random order each session. Results showed that the SHRs choose more impulsively than the WKYs, adding to the body of evidence suggesting that the SHR may be a suitable model of ADHD. Both models of choice fitted the impulsive choices of the SHRs and WKYs well; positive correlations between estimates of parameters k and s suggested compatibility between models of choice showing that impulsivity increases with training.

Limits to preference and the sensitivity of choice to rate and amount of food.

Studies of choice holding food-amount ratio constant while varying food-rate ratio within sessions showed that local changes in preference depend on relative amount of food. The present study investigated whether sensitivity of choice to food-rate ratio and sensitivity to food-amount ratio are independent of one another when food-rate ratios are varied across sessions and food-amount ratios are varied within sessions. Food deliveries for rats' presses on the left and right levers were scheduled according to three different food-rate ratios of 1:1, 9:1, and 1:9; each food-rate ratio lasted for 106 sessions and was arranged independently of seven food-amount ratios (7:1, 6:2, 5:3, 4:4, 3:5, 2:6, and 1:7 food pellets) occurring within sessions in random sequence. Each amount ratio lasted for 10 food deliveries and was separated from another by a 60-s blackout. Sensitivity to rate ratio was high (1.0) across food deliveries. Sensitivity to amount ratio was low when food rates were equal across alternatives, but was high when rate ratio and amount ratio opposed one another. When rate ratio and amount ratio went in the same direction, choice ratio reached an elevenfold limit which reduced sensitivity to approximately zero. We conclude that three factors affect sensitivity to amount: (1) the limit to preference, (2) the equal effect on preference of amounts greater than four pellets, and (3) the absence of differential effects of switches in amount in the equal-rates (1:1) condition. Taken together, these findings indicate that rate and amount only sometimes combine independently as additive variables to determine preference when amount ratios vary frequently within sessions.

A long-term study of the impulsive choices of Lewis and Fischer 344 rats.

Contemporary analyses of choice were implemented to analyze the acquisition and maintenance of response allocation in Lewis (LEW) and Fischer 344 (F344) rats. A concurrent-chains procedure varied the delay to the larger reinforcer (0.1, 5, 10, 20, 40, and 80 s). Delays were presented within sessions in ascending, descending, and random orders. Each condition lasted 105 days, and the entire data set was analyzed to obtain discounting functions for each block of 15 sessions and each food delivery across delay components. Both a hyperbolic-decay model and the generalized matching law described well the choices of LEW and F344 rats. Estimates of discounting rate and sensitivity to the immediacy of reinforcement correlated positively. The slope of the discounting function changed with presentation orders of the delays to the larger reinforcer. Extended training reduced differences between the LEW and F344 rats in discounting rates, sensitivity to the immediacy of reinforcement, and estimates of the area under the curve. We concluded that impulsive choice can change as a function of learning and is not a static property of behavior that is mainly determined by genetic and neurochemical mechanisms. Choosing impulsively may be an advantage for organisms searching for food in rapidly changing environments.

6-Hydroxydopamine lesions in the medial prefrontal cortex of rats exposed to a peak-interval procedure

Impaired temporal control is symptomatic of several neurological disorders; recently, it has been implicated in schizophrenia. An animal model of schizophrenia using 6-hydroxydopamine (6-OHDA) infused to the medial pre-frontal cortex (mPFC) was employed to examine its effects on temporal control. Twelve rats were trained on a peak-interval procedure (PIP) until stable patterns of behavior were obtained. Rats infused with 6-OHDA responded less during peak trials and their peak functions were flatter than sham rats. These results are consistent with similar studies with transgenic mice with increased striatal dopamine D2 receptor activity. Lesions in the mPFC decreased motivation to respond in a PIP. These effects may be considered analogous to negative symptoms of schizophrenia...

6-Hydroxydopamine lesions in the medial prefrontal cortex of rats exposed to a peak-interval procedure

Impaired temporal control is symptomatic of several neurological disorders; recently, it has been implicated in schizophrenia. An animal model of schizophrenia using 6-hydroxydopamine (6-OHDA) infused to the medial pre-frontal cortex (mPFC) was employed to examine its effects on temporal control. Twelve rats were trained on a peak-interval procedure (PIP) until stable patterns of behavior were obtained. Rats infused with 6-OHDA responded less during peak trials and their peak functions were flatter than sham rats. These results are consistent with similar studies with transgenic mice with increased striatal dopamine D2 receptor activity. Lesions in the mPFC decreased motivation to respond in a PIP. These effects may be considered analogous to negative symptoms of schizophrenia.(AU)

Effects of Haloperidol on Responding Maintained with Food and Sucrose-Water / Efectos del Haloperidol en Respuestas Mantenidas con Comida y Agua Azucarada

La dopamina es un mediador del valor hedonico de los reforzadores. El haloperidol reduce el valor del reforzador, interfiriendo con la emisión de conductas operantes. Alternativamente, interferencias en la transmisión de dopamina no impiden que los animales consuman el alimento. Los reforzadores quedan intactos después de la administración del haloperidol. Exploramos esas posibilidades con dos tipos de reforzadores, comida y agua azucarada, entregados en programas múltiples de reforzamiento. La comida mantuvo un mayor número de presiones de palanca que el agua-azucarada. Las presiones de palanca y los reforzadores obtenidos disminuyeron con el incremento en la dosis de haloperidol. Las diferencias en presiones de palanca entre alimento y agua-azucarada se mantuvieron con el haloperidol. La administración subcutánea del haloperidol causó mayores decrementos en las presiones de palanca que la administración intraperitoneal, pero no necesariamente causó decrementos en los reforzadores obtenidos; con el haloperidol las ratas continuaron produciendo los dos tipos de reforzadores.

The hedonic value of reinforcers is mediated by dopamine. Accordingly, haloperidol diminishes the value of reinforcers, by interfering with the emission of operant behaviors. Alternatively, the interference of dopamine transmission does not prevent animals from eating food. Thus, reinforcers remain intact after the administration of haloperidol. We assessed these possibilities with eight Wistar rats and two types of reinforcers, food-pellets and sucrose-water, delivered under multiple reinforcement schedules. In general, lever presses maintained by food-pellets were higher than those maintained by sucrose-water. Haloperidol produced dose-related decreases in lever presses and obtained reinforcers. Different doses had no effect on the number of lever presses. Subcutaneous administrations of haloperidol produced higher decreases in lever presses than intra-peritoneal administrations. Decreases in lever pressing were not necessarily accompanied by substantial reductions in obtained food-pellets and sucrose-water reinforcers; under the effects of haloperidol rats continued to produce a considerable number of both types of reinforcers.

Dynamics of choice: relative rate and amount affect local preference at three different time scales.

To examine extended control over local choice, the present study investigated preference in transition as food-rate ratio provided by two levers changed across seven components within daily sessions, and food-amount ratio changed across phases. Phase 1 arranged a food-amount ratio of 4:1 (i.e., the left lever delivered four pellets and the right lever one pellet); Phase 2 reversed the food-amount ratio to 1:4, and in Phase 3 the food-amount ratio was 3:2. At a relatively extended time scale, preference was described well by a linear relation between log response ratio and log rate ratio (the generalized matching law). A small amount of carryover occurred from one rate ratio to the next but disappeared after four food deliveries. Estimates of sensitivity to food-amount ratio were around 1.0 and were independent of rate ratio. Analysis across food deliveries within rate-ratio components showed that the effect of a small amount was diminished by the presence of a large amount-that is, when a larger amount was present in the situation (three or four pellets), the value of a small amount (one or two pellets) became paltry. More local analysis of visits to the levers between food deliveries showed that postfood visits following a large amount were disproportionately longer than following a small amount. Continuing food deliveries from the same source tended to make visits less dependent on relative amount, but a discontinuation (i.e., food from the other lever) reinstated dependence on relative amount. Analysis at a still smaller time scale revealed preference pulses following food deliveries that confirmed the tendency toward dependence on absolute amount with continuing deliveries, and toward dependence on relative amount following discontinuations. A mathematical model based on a linear-operator equation accounts for many of the results. The larger and longer preference following a switch to a larger amount is consistent with the idea that local preference depends on relatively extended variables even on short time scales.

Sensitivity, changeover responses, and choice in transition.

Studies of choice in steady state have shown that sensitivity to reinforcement increases with increasing fixed-ratio changeover (FR CO) requirements. We assessed the generality of this finding with choice in transition. Food deliveries were programmed according to concurrent variable-interval (VI) schedules. Seven different VI pairs arranged ratios of food deliveries (left/right) of 27:1, 9:1, 3:1, 1:1, 1:3, 1:9, and 1:27 at a constant overall rate across components. Within sessions, all seven ratios were presented in random order. Each component lasted for 10 food deliveries; components were separated by 60-s blackouts. A changeover lever required 1, 2, 4, 8, 16, 32, and 64 responses to alternate between two main levers. Redeterminations to all FR COs, but 64 responses, were obtained in descending order. Choice adjusted rapidly to rapid changes in the reinforcer ratio, tracking the lever associated with the highest probability of reinforcer. Sensitivity to reinforcement increased with increasing FR CO, replicating the negatively accelerated function found in our earlier study. With successive reinforcers in components, however, sensitivity reached asymptote values sooner with the largest (8, 16, and 32 responses), than with the smallest (1, 2, and 4 responses), FR CO requirements.

Dynamics of choice and frequency of environment change.

Recent research has examined the effects on behavioral adjustment of a range of within-session changes in the reinforcer ratio available from two alternatives. Regularities between extended patterns of behavior (choice) and a distribution of food deliveries (expressed as a food-rate ratio) have emerged from such studies. To assess the generality of these findings, the present study manipulated rats' experience in the choice situation. Sixteen rats were divided into two groups of eight rats each. Seven non-signaled food-rate ratios were arranged to occur for responses on two concurrently available levers. Group 1 experienced a different food-rate ratio every day; each ratio provided 70 food deliveries, and none of the seven ratios operated for 2 consecutive days. Group 2 experienced the same sequence of food-rate ratios, except that each food-rate ratio remained in effect for 17 consecutive days (providing 70 food deliveries per session), after which a different food-rate ratio was selected for the next 17 sessions. In all cases, preference was adjusted to changes in food-rate ratios. Estimates of sensitivity to changes in food-rate ratios increased with an increasing number of sessions (i.e., experience in the choice situation).

Effects of reinforcer quality and step size on rats' performance under progressive ratio schedules.

The Mathematical Principles of Reinforcement (MPR) model accurately predicts performance in fixed ratio (FR) schedules of reinforcement. The aim of the present study was to assess the generality of MPR with rats responding under progressive ratio (PR) schedules of different step sizes (PR1 or PR3) that provided either food or saccharin pellets. The results showed that the rats responding for saccharin pellets produced higher breakpoints (i.e., completed higher ratios) than those responding for food pellets. In terms of theoretical parameters, one finding unanticipated on the basis of MPR was that the a parameter (specific activation) was higher for the PR3 schedules. This finding suggests that specific activation may be affected indirectly by motor parameters of the task.

Haloperidol, dynamics of choice, and the parameters of the matching law.

The idea that dopamine mediates the reinforcing effects of stimuli persists in the field of neurosciences. The present study shows that haloperidol, a dopamine antagonist, does not eliminate the reinforcing value of food reinforcers. The ratio of reinforcers changed seven times across two levers within sessions, modeling a dynamic environment. The magnitude of the reinforcer was manipulated independently of the reinforcer ratio. Four doses of intraperitoneal haloperidol were assessed over periods of 12 daily sessions. Haloperidol did not impair the discrimination that the rats established between rich and lean levers; the response distributions favored the lever associated with the higher probability of reinforcement and the larger pellets. The parameters of the generalized matching law (bias and sensitivity) were used to estimate effects of haloperidol upon the motor system and upon the rats' motivation for food reinforcers.

Fix and sample with rats in the dynamics of choice.

The generality of the molar view of behavior was extended to the study of choice with rats, showing the usefulness of studying order at various levels of extendedness. Rats' presses on two levers produced food according to concurrent variable-interval variable-interval schedules. Seven different reinforcer ratios were arranged within each session, without cues identifying them, and separated by blackouts. To alternate between levers, rats pressed on a third changeover lever. Choice changed rapidly with changes in component reinforcer ratio, and more presses occurred on the lever with the higher reinforcer rate. With continuing reinforcers, choice shifted progressively in the direction of the reinforced lever, but shifted more slowly with each new reinforcer. Sensitivity to reinforcer ratio, as estimated by the generalized matching law, reached an average of 0.9 and exceeded that documented in previous studies with pigeons. Visits to the more-reinforced lever preceded by a reinforcer from that lever increased in duration, while all visits to the less-reinforced lever decreased in duration. Thus, the rats' performances moved faster toward fix and sample than did pigeons' performances in previous studies. Analysis of the effects of sequences of reinforcer sources indicated that sequences of five to seven reinforcers might have sufficed for studying local effects of reinforcers with rats. This study supports the idea that reinforcer sequences control choice between reinforcers, pulses in preference, and visits following reinforcers.

Evaluando haloperidol en ratas con el paradigma de la barrera de selección / Assessing haloperidol in rats with the barrier choice paradigm

Diez ratas eligieron entre dos palancas que concurrentemente proporcionaron alimento de acuerdo a dos programas de intervalo aleatorio. La situación de elección se modificó para incluir desplazamiento físico (viaje). Se colocó una barrera de 15.2 cm., entre las dos palancas. Durante treinta días consecutivos las ratas escalaron la barrera viajando entre las palancas. Después de inyectaron (ip) cuatro dosis de haloperidol. En pasos idénticos de 15.2 cm., la barrera se incrementó hasta llegar a una altura de 76 cm. Con cada barrera se repitió el procedimiento de administración de droga. Las duraciones de presión de palanca y los tiempos entre respuestas aumentaron con incrementos en las dosis de haloperidol. Las escaladas en la barrera decrementaron con incrementos en las dosis de haloperidol y en la altura de la barrera. Con las dos manipulaciones as ratas permanecieron más tiempo en las palancas, presionaron más las palancas y obtuvieron más comida. Las funciones de retroalimentación de reforzamiento cambiaron en función de la altura de la barrera y de la dosis de haloperidol sugiriendo efectos similares en las presiones de palanca. Las ratas fueron más eficientes en producir comida cuando tuvieron que escalar las barreras más altas y bajo los efectos de las dosis más altas de haloperidol. En general, los resultados confirmaron que el haloperidol afecta al sistema motriz, pero no se encontró evidencia que indique que esta droga afecta al sistema motivacional.