Modeling VI and VDRL feedback functions: Searching normative rules through computational simulation.

We present the mathematical description of feedback functions of variable interval and variable differential reinforcement of low rates as functions of schedule size only. These results were obtained using an R script named Beak, which was built to simulate rates of behavior interacting with simple schedules of reinforcement. Using Beak, we have simulated data that allow an assessment of different reinforcement feedback functions. This was made with unparalleled precision, as simulations provide huge samples of data and, more importantly, simulated behavior is not changed by the reinforcement it produces. Therefore, we can vary response rates systematically. We've compared different reinforcement feedback functions for random interval schedules, using the following criteria: meaning, precision, parsimony, and generality. Our results indicate that the best feedback function for the random interval schedule was published by Baum (1981). We also propose that the model used by Killeen (1975) is a viable feedback function for the random differential reinforcement of low rates schedule. We argue that Beak paves the way for greater understanding of schedules of reinforcement, addressing still open questions about quantitative features of simple schedules. Also, Beak could guide future experiments that use schedules as theoretical and methodological tools.

Neonatal exposure to ultrafine iron but not combined iron and sulfur aerosols recapitulates air pollution-induced impulsivity in mice.

Air pollution (AP) is becoming recognized as a major threat to neurological health across the lifespan with increased risk of both neurodevelopmental and neurodegenerative disorders. AP is a complex mixture of gases and particulate matter, with adsorbed contaminants including metals and trace elements, which may differentially contribute to its neurodevelopmental impacts. Iron (Fe) is one of the most abundant metals found in AP, and Fe concentrations may drive some behavioral deficits observed in children. Furthermore, brains of neonate mice exposed to concentrated ambient ultrafine particulate matter (UFP) show significant brain accumulation of Fe and sulfur (S) supporting the hypothesis that AP exposure may lead to brain metal dyshomeostasis. The current study determined the extent to which behavioral effects of UFP, namely memory deficits and impulsive-like behavior, could be recapitulated with exposure to Fe aerosols with or without concomitant SO2. Male and female neonate mice were either exposed to filtered air or spark discharge-generated ultrafine Fe particles with or without SO2 gas (n = 12/exposure/sex). Inhalation exposures occurred from postnatal day (PND) 4-7 and 10-13 for 4 hr/day, mirroring our previous UFP exposures. Mice were aged to adulthood prior to behavioral testing. While Fe or Fe + SO2 exposure did not affect gross locomotor behavior, Fe + SO2-exposed females displayed consistent thigmotaxis during locomotor testing. Neither exposure affected novel object memory. Fe or Fe + SO2 exposure produced differential outcomes on a fixed-interval reinforcement schedule with males showing higher (Fe-only) or lower (Fe + SO2) response rates and postreinforcement pauses (PRP) and females showing higher (Fe-only) PRP. Lastly, Fe-exposed, but not Fe + SO2-exposed, males showed increased impulsive-like behavior in tasks requiring response inhibition with no such effects in female mice. These findings suggest that: 1) exposure to realistic concentrations of Fe aerosols can recapitulate behavioral effects of UFP exposure, 2) the presence of SO2 can modulate behavioral effects of Fe inhalation, and 3) brain metal dyshomeostasis may be an important factor in AP neurotoxicity.

An assessment of executive function in two different rat models of attention-deficit hyperactivity disorder: Spontaneously hypertensive versus Lphn3 knockout rats.

Attention-deficit/hyperactivity disorder (ADHD) a common neurodevelopmental disorder of childhood and often comorbid with other externalizing disorders (EDs). There is evidence that externalizing behaviors share a common genetic etiology. Recently, a genome-wide, multigenerational sample linked variants in the Lphn3 gene to ADHD and other externalizing behaviors. Likewise, limited research in animal models has provided converging evidence that Lphn3 plays a role in EDs. This study examined the impact of Lphn3 deletion (i.e., Lphn3-/- ) in rats on measures of behavioral control associated with externalizing behavior. Impulsivity was assessed for 30 days via a differential reinforcement of low rates (DRL) task and working memory evaluated for 25 days using a delayed spatial alternation (DSA) task. Data from both tasks were averaged into 5-day testing blocks. We analyzed overall performance, as well as response patterns in just the first and last blocks to assess acquisition and steady-state performance, respectively. "Positive control" measures on the same tasks were measured in an accepted animal model of ADHD-the spontaneously hypertensive rat (SHR). Compared with wildtype controls, Lphn3-/- rats exhibited deficits on both the DRL and DSA tasks, indicative of deficits in impulsive action and working memory, respectively. These deficits were less severe than those in the SHRs, who were profoundly impaired on both tasks compared with their control strain, Wistar-Kyoto rats. The results provide evidence supporting a role for Lphn3 in modulating inhibitory control and working memory, and suggest additional research evaluating the role of Lphn3 in the manifestation of EDs more broadly is warranted.

The ventral hippocampus CA3 is critical in regulating timing uncertainty in temporal decision-making.

Timing uncertainty is a critical component of temporal decision-making, as it determines the decision strategies that maximize reward rate. However, little is known about the biological substrates of timing uncertainty. In this study, we report that the CA3 subregion of the ventral hippocampus (vCA3), a relatively unexplored area in timing, is critical in regulating timing uncertainty that informs temporal decision making. Using a variant of the differential reinforcement of low rates of responding (DRL) task that incorporates differential levels of approach-avoidance conflict, rats were trained to wait a minimum of 6 s to earn a reward that was paired with varying durations of foot shock. Post-training chemogenetic inhibition of the vCA3 reduced timing uncertainty without affecting mean wait times, irrespective of the level of conflict experienced. Simulations based on the information-processing variant of scalar expectancy theory (SET) revealed that the vCA3 may be important in modulating decision threshold or switch closure latency variability.

Temporal regulation of children with autism spectrum disorder exposed to a differential-reinforcement-of low-rates schedule.

This study investigated temporal adjustment of children with autism spectrum disorder under a differential-reinforcement-of-low-rates (DRL) schedule. Sixteen participants, aged 3.2 to 7 years, were exposed to two conditions, DRL 5 s and DRL 20 s. Children participated in 7 sessions in each condition, except for 1 participant who attained the adjustment criteria in the DRL 5-s schedule. Temporal adjustment was measured with the proportion of reinforced interresponse times (IRTs) and the mean IRT. The operant response was a press on a touch screen and the reinforcers were cartoons. IQ and receptive language were measured prior to the DRL sessions. Results showed that the mean proportion of reinforced IRTs was slightly higher in the DRL 5-s schedule. The mean IRT was above the IRT requirement in both conditions. However, substantial individual variability was observed. Children with higher IQ and receptive language scores presented a greater proportion of reinforced IRTs in both conditions. Moreover, participants who adjusted their responses to the DRL 5-s schedule were more likely to adjust responding to the DRL 20-s schedule. This suggests that some children might be more sensitive to reinforcement contingencies than others. This study points at future research in the field of timing in children.

Differential reinforcement-of-low-rate procedures: A systematic replication with students with autism spectrum disorder.

Becraft, Borrero, Davis, Mendres-Smith, and Castillo (2018) studied the effects of two different types of DRL schedules (full session and spaced responding) under 2 sets of stimulus conditions (with and without signals). Reduced rates of responding maintained under both types of DRL schedules, when signals were included. The present study represents a replication of procedures by Becraft et al. involving learners with autism spectrum disorder (ASD). The results replicated those of Becraft et al. in that responding in both full-session and spaced-responding DRL schedules was low, but not eliminated. These results provide preliminary evidence to suggest that children with ASD are responsive to signals in DRL arrangements, which may set the stage for evaluation of signaled DRL arrangements for socially significant response forms.

Ethanol Reinforcement Elicits Novel Response Inhibition Behavior in a Rat Model of Ethanol Dependence.

Lower impulse control is a known risk factor for drug abuse vulnerability. Chronic experience with illicit drugs is suggested to enhance impulsivity and thereby perpetuate addiction. However, the nature of this relationship (directionality, causality) with regard to alcohol use disorder is unclear. The present study tested the hypothesis that higher impulsivity is observed during chronic intermittent ethanol vapor inhalation (CIE; a model of ethanol dependence) and subsequent abstinence from CIE in adult Wistar rats. Impulsivity was tested using a differential reinforcement of low rates 15 s (DRL15) schedule using either nondrug reward (palatable modified sucrose pellets) or sweetened ethanol. A decrease in the efficiency of earning reinforcers (expressed as % reinforcers/responses) is indicative of a decrease in response inhibition or an increase in impulsivity. The efficiency of reinforcement and amount of reinforcers earned were unaltered in CIE and control animals when the reinforcer was sucrose. When the reinforcer was sweetened ethanol, the efficiency of reinforcement increased in CIE rats compared with controls only during protracted abstinence. Responding for sweetened ethanol under a progressive-ratio schedule was more rapid in CIE rats during protracted abstinence. Contrary to the initial hypothesis, impulsivity did not increase in rats with a history of CIE; instead, it decreased when ethanol was used as the reinforcer. Furthermore, although the efficiency of ethanol reinforcement did not differ between CIE and control animals during CIE, CIE rats escalated the amount of sweetened ethanol consumed, suggesting that behavioral adaptations that are induced by CIE in rats that are tested under a DRL15 schedule appear to be targeted toward the maximization of ethanol intake and thus may contribute to escalation and relapse.

The role of signals in two variations of differential-reinforcement-of-low-rate procedures.

Differential-reinforcement-of-low-rate (DRL) schedules are used to decrease the overall rate of, but not eliminate, a target response. Two variations of DRL, spaced-responding and full-session, exist. Preliminary comparative analyses suggest that the two schedules function differently when unsignaled. We compared response rates under these two DRL variations with and without signals. In Experiment 1, five preschool students played a game in which points were earned under DRL schedules. In some sessions, a stimulus signaled when responses would be reinforced (S+) or not reinforced (S-). In others, only an S- was present. Signals (S+/S-) facilitated and maintained responding in both types of DRL schedules. In Experiment 2, we modified the signals with five different preschoolers. Instead of an S- only, we did not present any signals. Elimination and high variability of the target response were observed with the S- only and absence of S+/S-, respectively. Signaled DRL schedules are recommended for application.

Differential Reinforcement of Low Rate Schedules Reduce Severe Problem Behavior.

Differential-reinforcement-of-low-rate (DRL) schedules are reinforcement contingencies designed to reduce response rates. A common variation of the DRL arrangement is known as full-session DRL ( f-DRL), in which a reinforcer is presented at the end of an interval if the response rate during that interval is below a predetermined criterion. Prior human operant research involving arbitrary mouse clicks has shown that the f-DRL is likely to reduce target responding to near zero rates. Similarly, applied research has shown that the f-DRL is likely to reduce minimally disruptive classroom behavior. There are, however, relatively few successful applications of the f-DRL to severe forms of problem behavior (e.g., self-injurious behavior). Thus, the purpose of this study was to examine the effects of f-DRL on the severe problem behavior of individuals with intellectual and developmental disabilities. For four participants, the f-DRL reduced severe problem behavior by clinically significant levels. Furthermore, results of a contingency strength analysis showed a strong negative contingency strength between target responding and reinforcer delivery for all participants.

Adolescent methylphenidate treatment differentially alters adult impulsivity and hyperactivity in the Spontaneously Hypertensive Rat model of ADHD.

Impulsivity and hyperactivity are two facets of attention deficit/hyperactivity disorder (ADHD). Impulsivity is expressed as reduced response inhibition capacity, an executive control mechanism that prevents premature execution of an intermittently reinforced behavior. During methylphenidate treatment, impulsivity and hyperactivity are decreased in adolescents with ADHD, but there is little information concerning levels of impulsivity and hyperactivity in adulthood after adolescent methylphenidate treatment is discontinued. The current study evaluated impulsivity, hyperactivity as well as cocaine sensitization during adulthood after adolescent methylphenidate treatment was discontinued in the Spontaneously Hypertensive Rat (SHR) model of ADHD. Treatments consisted of oral methylphenidate (1.5mg/kg) or water vehicle provided Monday-Friday from postnatal days 28-55. During adulthood, impulsivity was measured in SHR and control strains (Wistar Kyoto and Wistar rats) using differential reinforcement of low rate (DRL) schedules. Locomotor activity and cocaine sensitization were measured using the open-field assay. Adult SHR exhibited decreased efficiency of reinforcement under the DRL30 schedule and greater levels of locomotor activity and cocaine sensitization compared to control strains. Compared to vehicle, methylphenidate treatment during adolescence reduced hyperactivity in adult SHR, maintained the lower efficiency of reinforcement, and increased burst responding under DRL30. Cocaine sensitization was not altered following adolescent methylphenidate in adult SHR. In conclusion, adolescent treatment with methylphenidate followed by discontinuation in adulthood had a positive benefit by reducing hyperactivity in adult SHR rats; however, increased burst responding under DRL compared to SHR given vehicle, i.e., elevated impulsivity, constituted an adverse consequence associated with increased risk for cocaine abuse liability.

Temporal control of behaviour in children with differential reinforcement of low rates schedule: the role of age, language and cognitive functioning on temporal regulation.

Research on temporal regulation in children has been prolific until early 1990s and has received a very limited attention since then. However, the studies focussed mainly on very short durations, and many questions raised at that time remain unanswered (Clément et al., 2007). The scope of this study was to evaluate temporal control in children with differential reinforcement of low-rates (DRL) schedule. Objectives were (a) to evaluate the performance in DRL with two distinct durations; (b) to evaluate the relationship between performance, IQ and language; and (c) to observe children's response patterns across the sessions. Eleven children aged from 2.6 to 7 years old were exposed to a DRL 5s and a DRL 20s schedule. No significant correlation was observed between language, IQ and the performance in DRL. In DRL 5s, seven children adjusted their responses and six in DRL 20s. Age was positively correlated to performance in DRL 5s, while the response patterns in DRL 20s were hardly predictable. In both conditions, children aged from 4.6 years old showed a lower proportion of bursting responses, a lower rate of response, a larger proportion of reinforced responses and a higher optimisation coefficient.

A laboratory comparison of two variations of differential-reinforcement-of-low-rate procedures.

We compared 2 variations of differential-reinforcement-of-low-rate (DRL) procedures: spaced-responding DRL, in which a reinforcer was delivered contingent on each response if a specified interval had passed since the last response, and full-session DRL, in which a reinforcer was presented at the end of an interval if the response rate was below criterion within the specified interval. We used a human-operant procedure and analyzed within-session responding to assess any similarities or differences between procedures. Data revealed a positive contingency between responding and reinforcement under the spaced-responding DRL schedule and a negative contingency under the full-session DRL schedule. Furthermore, 60% of the participants discontinued responding by the last full-session DRL session. Implications for the appropriate procedural and taxonomical usage of both DRL schedules are discussed.

Using differential reinforcement of low rates to reduce children's requests for teacher attention.

We evaluated the effectiveness of full-session differential reinforcement of low rates of behavior (DRL) on 3 primary school children's rates of requesting attention from their teacher. Using baseline rates of responding and teacher recommendations, we set a DRL schedule that was substantially lower than baseline yet still allowed the children access to teacher assistance. The DRL schedule was effective in reducing children's requests for assistance and approval, and the teacher found the intervention highly useful and acceptable. The possible mechanisms that account for behavior change using full-session DRL schedules are discussed.

História comportamental: efeitos de história de reforçamento em FR sobre desempenho posterior em DRL em ratos Wistar / Behavioral history: effects of a reinforcement historu under FR over an ulterior performece under DRL in Wistar rats / Historia conductual: efectos de la historia de refuerzo en FR acerca de una actuación posterior en DRL en ratas Wistar

Este trabalho pretende, inicialmente, apontar a importância dos estudos de história comportamental, assim como as definições necessárias à sua compreensão. Em seguida, apresentam-se algumas das mais relevantes pesquisas na área e algumas das lacunas deixadas por elas. Para discutir tais lacunas, um experimento foi realizado com o objetivo de verificar: (i) a existência ou não de alterações no desempenho de sujeitos sob esquema de reforçamento DRL, tendo eles passado por uma história anterior sob esquema de reforçamento FR; (ii) a transitoriedade de tais efeitos; e (iii) possíveis relações que permitem que esses efeitos ocorram. Nesse sentido, o experimento foi desenvolvido com ratos Wistar, submetidos a diferentes histórias de reforçamento - quatro sujeitos foram divididos em dois grupos, utilizando-se esquemas DRL, FR ou a ausência de submissão a um esquema. Os resultados demonstraram que as taxas de respostas sob esquema DRL, após história de FR, foram superiores às taxas desses mesmos sujeitos antes da submissão ao esquema FR; esses efeitos não pareceram ser transitórios, pelo menos na amostra analisada. Com base nos resultados, argumenta-se que os efeitos da história poderiam reaparecer em situações posteriores, mesmo após um período em que parecem ter sido apagados. Discute-se, finalmente, a possibilidade de que esses efeitos tornem-se evidentes em decorrência da fragilidade do esquema atual, bem como as possíveis aplicações e implicações da pesquisa para a área da análise do comportamento.

This paper initially intends to point out the importance of the behavioral history studies to Behaviorism, as well as the definitions necessary to its understanding. Secondly, it presents some of the most relevant researches in the area and some of the gaps they have left. In order to discuss such gaps, an experiment is conducted with the objective of verifying: (i) the existence or not of changes in the performance of a subject under a DRL reinforcement schedule, having him passed through a previous history under FR reinforcement schedule; (ii) the transience of such effects; and (iii) possible relations which let these effects to occur. In this sense, the experiment was developed using rats, which were subjected to different reinforcement histories - four subjects were divided into two groups and subjected either to DRL, FR or no reinforcement schedules. The results demonstrated that the response rated under DRL schedule, after a reinforcement history under FR, were superior to those rates of the same subjects before their subjection to the FR schedule; these effects seemed not to be transient, at least on the analyzed sample. Based on the results, it is argued that the effects of history might reappear in ulterior situations, even after a period in which they seem to have been erased. The possibility that these effects occur due to the weakness of the current schedule, as well as the possible applications and implications of the research to the general area of Behavior Analysis are also discussed.

Este trabajo inicialmente tiene la intención de señalar la importancia de los estudios de la historia del comportamiento para el conductismo, así como las definiciones necesarias para su comprensión. En segundo lugar, se presentan algunos de las más importantes investigaciones en el área y algunos de los vacíos que han dejado. Para discutir estas lagunas, se lleva a cabo un experimento con el objetivo de verificar: (i) la existencia o no de los cambios en el desempeño de un sujeto en virtud de un programa de reforzamiento DRL, habiendo pasado por una historia anterior en aplicación de programa de reforzamiento FR; (ii) la fugacidad de estos efectos; y (iii) posibles relaciones que permiten que estos efectos se produzcan. En este sentido, el experimento se desarrolló con ratas Winstar, sometidas a diferentes historias de refuerzo - cuatro sujetos fueron divididos en dos grupos sometidos a programa de reforzamiento DRL, FR o a ningún programa. Los resultados han demostrado que las tasas de respuesta en el programa de reforzamiento DRL, después de una historia de refuerzo debajo de FR, fueran superiores a las tasas de los mismos sujetos antes de su sometimiento al programa FR; estos efectos no parecen ser transitorios, al menos en la muestra analizada. Basándose en los resultados, se argumenta que los efectos de la historia podrían volver a aparecer en situaciones ulteriores, incluso después de un período en el que parecen haber sido borrados. Son discutidas también la posibilidad de que estos efectos se evidencien debido a la debilidad de la programación actual, así como las posibles aplicaciones e implicaciones de la investigación para el área del análisis de la conducta.