ABSTRACT
Revaluation refers to phenomena in which the strength of an operant is altered by reinforcer-related manipulations that take place outside the conditioning situation in which the operant was selected. As an example, if lever pressing is acquired using food as a reinforcer and food is later paired with an aversive stimulus, the frequency of lever pressing decreases when subsequently tested. Associationist psychology infers from such findings that conditioning produces a response-outcome (i.e., reinforcer) association and that the operant decreased in strength because pairing the reinforcer with the aversive stimulus changed the value of the outcome. Here, we present an approach to the interpretation of these and related findings that employs neural network simulations grounded in the experimental analysis of behavior and neuroscience. In so doing, we address some general issues regarding the relations among behavior analysis, neuroscience, and associationism.
Subject(s)
Behavior, Animal/physiology , Nerve Net/physiology , Animals , Conditioning, Operant/physiology , Feedback , Humans , Reinforcement, PsychologyABSTRACT
The central focus of this essay is whether the effect of reinforcement is best viewed as the strengthenng of responding or the strengthening of the environmental control of responding. We make the argument that adherence to Skinner's goal of achieving a moment-to-moment analysis of behavior compels acceptance of the latter view. Moreover, a thoroughgoing commitment to a moment-to-moment analysis undermines the fundamental distinction between the conditioning process instantiated by operant and respondent contingencies while buttressing the crucially important differences in their cumulative outcomes. Computer simulations informed by experimental analyses of behavior and neuroscience are used to illustrate these points.
Subject(s)
Association Learning , Computer Simulation , Conditioning, Classical , Conditioning, Operant , Neural Networks, Computer , Animals , Association Learning/physiology , Conditioning, Classical/physiology , Conditioning, Operant/physiology , Discrimination Learning/physiology , Dopamine/physiology , Hippocampus/physiology , Humans , Reinforcement Schedule , Ventral Tegmental Area/physiologyABSTRACT
We describe a principle of reinforcement that draws upon experimental analyses of both behavior and the neurosciences. Some of the implications of this principle for the interpretation of behavior are explored using computer simulations of adaptive neural networks. The simulations indicate that a single reinforcement principle, implemented in a biologically plausible neural network, is competent to produce as its cumulative product networks that can mediate a substantial number of the phenomena generated by respondent and operant contingencies. These include acquisition, extinction, reacquisition, conditioned reinforcement, and stimulus-control phenomena such as blocking and stimulus discrimination. The characteristics of the environment-behavior relations selected by the action of reinforcement on the connectivity of the network are consistent with behavior-analytic formulations: Operants are not elicited but, instead, the network permits them to be guided by the environment. Moreover, the guidance of behavior is context dependent, with the pathways activated by a stimulus determined in part by what other stimuli are acting on the network at that moment. In keeping with a selectionist approach to complexity, the cumulative effects of relatively simple reinforcement processes give promise of simulating the complex behavior of living organisms when acting upon adaptive neural networks.
Subject(s)
Behavior , Conditioning, Operant , Reinforcement, Psychology , Biological Evolution , Biological Factors , Brain/physiology , Female , Humans , Male , Motor Cortex/physiology , Neural Pathways/physiology , Selection, GeneticABSTRACT
Contingencies of selection, be they phylogenetic or ontogenetic, merely set boundaries on units; they do not provide blueprints. Thus, variability is fundamental to all products of selection. Skinner, by characterizing the units of analysis in behavior as generic in nature, established his science squarely within the selectionist paradigm, thereby avoiding the tendency, common throughout psychology, to slip into essentialist analyses. The distinction between essentialism and selectionism is refined in this article, and prominent examples of essentialism in linguistics, theories of memory, theories of representation, associationism, and even in behavior analysis are identified. Recent trends in cognitive science--specifically, research on adaptive networks--is amenable to a selectionist interpretation, suggesting the possibility of future fruitful interactions with behavior analysis.
Subject(s)
Behaviorism , Biological Evolution , Cognition , Psychological Theory , Selection, Genetic , HumansABSTRACT
Explicit consolidation of memory, or fixation of declarative belief, appears to be physically represented in changes of synaptic conductances of neurons in the parietal-temporal-occipital association cortex (PTO) of the mammalian forebrain. This fixation of belief in PTO is postulated to be critically dependent on a diffuse reinforcement signal via the inferior temporal cortex (ITC) ultimately caused by an increased output of the CA1 pyramidal cells of hippocampus. Analogous to the reinforcing mechanisms of other forebrain systems, this updating of the connection weights of the neural nets in PTO by the output of the critical neurons in CA1 is directly related to concentrations of dopamine (DA). We propose that the delusions (i.e., unreasonable beliefs) of paranoid schizophrenia are caused by a hyperactivity of the same DA-sensitive CA1 neurons that are responsible for the fixation of normal beliefs. The dramatic reduction in delusions with administration of neuroleptics, as DA D2 blockers, in schizophrenics may thus be explained by their acting to ameliorate the hyperactivity of these CA1 DA D2 receptors.
Subject(s)
Dopamine/physiology , Hippocampus/physiopathology , Receptors, Dopamine/physiology , Schizophrenia, Paranoid/physiopathology , Schizophrenic Psychology , Antipsychotic Agents/therapeutic use , Brain Damage, Chronic/diagnosis , Brain Damage, Chronic/drug therapy , Brain Damage, Chronic/physiopathology , Brain Damage, Chronic/psychology , Brain Mapping , Hippocampus/drug effects , Humans , Neurons/drug effects , Neurons/physiology , Receptors, Dopamine/drug effects , Receptors, Dopamine D2 , Schizophrenia, Paranoid/diagnosis , Schizophrenia, Paranoid/drug therapy , Schizophrenia, Paranoid/psychologyABSTRACT
When discriminated interresponse-time (IRT) procedures have been used to assess preference relations among temporally extended operants, deviations from matching have been obtained. Using a yoked-control procedure, the present study found that key pecking in a discriminated IRT procedure has two sources of strength--that arising from the response-reinforcer contingency that is explicitly arranged, and that arising from a stimulus-reinforcer contingency that is a by-product of the explicitly arranged contingency. The key pecking of all lead birds, and that of 3 of the 4 birds exposed to a yoked autoshaping procedure, was controlled by the keylight that signaled the lead birds' criterion IRTs. Because stimulus control of key pecking by the keylight, whether autoshaped or discriminative, fosters deviations from matching, the discriminated IRT procedure does not provide an appropriate basis for conclusions about preference relations among IRTs.
Subject(s)
Conditioning, Operant , Discrimination Learning , Time Perception , Animals , Columbidae , Reinforcement Schedule , Transfer, PsychologyABSTRACT
A formal statement of a relational principle of reinforcement is developed that makes contact with analyses of choice, interresponse-time distributions, and stimulus control. Some implications for current theoretical and empirical work in the various areas are examined.
