Unified-theory-of-reinforcement neural networks do not simulate the blocking effect.

For the last 20 years the unified theory of reinforcement (Donahoe et al., 1993) has been used to develop computer simulations to evaluate its plausibility as an account for behavior. The unified theory of reinforcement states that operant and respondent learning occurs via the same neural mechanisms. As part of a larger project to evaluate the operant behavior predicted by the theory, this project was the first replication of neural network models based on the unified theory of reinforcement. In the process of replicating these neural network models it became apparent that a previously published finding, namely, that the networks simulate the blocking phenomenon (Donahoe et al., 1993), was a misinterpretation of the data. We show that the apparent blocking produced by these networks is an artifact of the inability of these networks to generate the same conditioned response to multiple stimuli. The piecemeal approach to evaluate the unified theory of reinforcement via simulation is critiqued and alternatives are discussed.

Against matching theory: predictions of an evolutionary theory of behavior dynamics.

A selectionist theory of adaptive behavior dynamics instantiates the idea that behavior evolves in response to selection pressure from the environment in the form of resource acquisition or threat escape or avoidance. The theory is implemented by a computer program that creates an artificial organism and animates it with a population of potential behaviors. The population undergoes selection, recombination, and mutation across generations, or ticks of time, which produces a continuous stream of behavior that can be studied as if it were the behavior of a live organism. Novel predictions of the evolutionary theory can be compared to predictions of matching theory in a critical experiment that arranges concurrent schedules with reinforcer magnitudes that vary across conditions in one component of the schedules but not the other. Matching theory and the evolutionary theory make conflicting predictions about the outcome of this critical experiment, such that the results must disconfirm at least one of the theories.

Selection dynamics in joint matching to rate and magnitude of reinforcement.

Virtual organisms animated by a selectionist theory of behavior dynamics worked on concurrent random interval schedules where both the rate and magnitude of reinforcement were varied. The selectionist theory consists of a set of simple rules of selection, recombination, and mutation that act on a population of potential behaviors by means of a genetic algorithm. An extension of the power function matching equation, which expresses behavior allocation as a joint function of exponentiated reinforcement rate and reinforcer magnitude ratios, was fitted to the virtual organisms' data, and over a range of moderate mutation rates was found to provide an excellent description of their behavior without residual trends. The mean exponents in this range of mutation rates were 0.83 for the reinforcement rate ratio and 0.68 for the reinforcer magnitude ratio, which are values that are comparable to those obtained in experiments with live organisms. These findings add to the evidence supporting the selectionist theory, which asserts that the world of behavior we observe and measure is created by evolutionary dynamics.

Development and preliminary validation of a behavioral task of negative reinforcement underlying risk-taking and its relation to problem alcohol use in college freshmen.

BACKGROUND: A long line of theoretical and empirical evidence implicates negative reinforcement as a process underlying the etiology and maintenance of risky alcohol use behaviors from adolescence through emerging adulthood. However, the bulk of this literature has relied on self-report measures, and there is a notable absence of behavioral modes of assessments of negative reinforcement-based alcohol-related risk-taking. To address this clear gap in the literature, the current study presents the first published data on the reliability and validity of the Maryland Resource for the Behavioral Utilization of the Reinforcement of Negative Stimuli (MRBURNS), which is a modified version of the positive reinforcement-based Balloon Analogue Risk Task (BART). METHODS: Participants included a convenience sample of 116 college freshmen ever regular drinkers (aged 18 to 19) who completed both behavioral tasks; self-report measures of negative reinforcement/avoidance constructs and of positive reinforcement/appetitive constructs to examine convergent validity and discriminant validity, respectively; and self-report measures of alcohol use, problems, and motives to examine criterion validity. RESULTS: The MRBURNS evidenced sound experimental properties and reliability across task trials. In support of convergent validity, risk-taking on the MRBURNS correlated significantly with negative urgency, difficulties in emotion regulation, and depressive and anxiety-related symptoms. In support of discriminant validity, performance on the MRBURNS was unrelated to risk-taking on the BART, sensation seeking, and trait impulsivity. Finally, pertaining to criterion validity, risk-taking on the MRBURNS was related to alcohol-related problems but not heavy episodic alcohol use. Notably, risk-taking on the MRBURNS was associated with negative reinforcement-based but not with positive reinforcement-based drinking motives. CONCLUSIONS: Data from this initial investigation suggest the utility of the MRBURNS as a behavioral measure of negative reinforcement-based risk-taking that can provide a useful complement to existing self-report measures to improve our understanding of the relationship between avoidant reinforcement processes and risky alcohol use.