Direct Approach or Detour: A Comparative Model of Inhibition and Neural Ensemble Size in Behavior Selection.

Organisms must cope with different risk/reward landscapes in their ecological niche. Hence, species have evolved behavior and cognitive processes to optimally balance approach and avoidance. Navigation through space, including taking detours, appears also to be an essential element of consciousness. Such processes allow organisms to negotiate predation risk and natural geometry that obstruct foraging. One aspect of this is the ability to inhibit a direct approach toward a reward. Using an adaptation of the well-known detour paradigm in comparative psychology, but in a virtual world, we simulate how different neural configurations of inhibitive processes can yield behavior that approximates characteristics of different species. Results from simulations may help elucidate how evolutionary adaptation can shape inhibitive processing in particular and behavioral selection in general. More specifically, results indicate that both the level of inhibition that an organism can exert and the size of neural populations dedicated to inhibition contribute to successful detour navigation. According to our results, both factors help to facilitate detour behavior, but the latter (i.e., larger neural populations) appears to specifically reduce behavioral variation.

The Missing Link Between Memory and Reinforcement Learning.

Reinforcement learning systems usually assume that a value function is defined over all states (or state-action pairs) that can immediately give the value of a particular state or action. These values are used by a selection mechanism to decide which action to take. In contrast, when humans and animals make decisions, they collect evidence for different alternatives over time and take action only when sufficient evidence has been accumulated. We have previously developed a model of memory processing that includes semantic, episodic and working memory in a comprehensive architecture. Here, we describe how this memory mechanism can support decision making when the alternatives cannot be evaluated based on immediate sensory information alone. Instead we first imagine, and then evaluate a possible future that will result from choosing one of the alternatives. Here we present an extended model that can be used as a model for decision making that depends on accumulating evidence over time, whether that information comes from the sequential attention to different sensory properties or from internal simulation of the consequences of making a particular choice. We show how the new model explains both simple immediate choices, choices that depend on multiple sensory factors and complicated selections between alternatives that require forward looking simulations based on episodic and semantic memory structures. In this framework, vicarious trial and error is explained as an internal simulation that accumulates evidence for a particular choice. We argue that a system like this forms the "missing link" between more traditional ideas of semantic and episodic memory, and the associative nature of reinforcement learning.

From Focused Thought to Reveries: A Memory System for a Conscious Robot.

We introduce a memory model for robots that can account for many aspects of an inner world, ranging from object permanence, episodic memory, and planning to imagination and reveries. It is modeled after neurophysiological data and includes parts of the cerebral cortex together with models of arousal systems that are relevant for consciousness. The three central components are an identification network, a localization network, and a working memory network. Attention serves as the interface between the inner and the external world. It directs the flow of information from sensory organs to memory, as well as controlling top-down influences on perception. It also compares external sensations to internal top-down expectations. The model is tested in a number of computer simulations that illustrate how it can operate as a component in various cognitive tasks including perception, the A-not-B test, delayed matching to sample, episodic recall, and vicarious trial and error.

Anticipatory models in gaze control: a developmental model.

Infants gradually learn to predict the motion of moving targets and change from a strategy that mainly depends on saccades to one that depends on anticipatory control of smooth pursuit. A model is described that combines three types of mechanisms for gaze control that develops in a way similar to infants. Initially, gaze control is purely reactive, but as the anticipatory models become more accurate, the gain of the pursuit will increase and lead to a larger fraction of smooth eye movements. Finally, a third system learns to predict changes in target motion, which will lead to fast retuning of the parameters in the anticipatory model.