On the neural substrates for exploratory dynamics in basal ganglia: a model.

We present a neural network model of basal ganglia that departs from the classical Go/NoGo picture of the function of its key pathways-the direct pathway (DP) and the indirect pathway (IP). In classical descriptions of basal ganglia function, the DP is known as the Go pathway since it facilitates movement and the IP is called the NoGo pathway since it inhibits movement. Between these two regimes, in the present model, we posit that there is a third Explore regime, which denotes random exploration of the space of actions. The proposed model is instantiated in a simple action selection task. Striatal dopamine is assumed to switch between DP and IP activation. The IP is modeled as a loop of the subthalamic nucleus (STN) and the globus pallidus externa (GPe), capable of producing chaotic activity. Simulations reveal that, while the system displays Go and NoGo regimes for extreme values of dopamine, at intermediate values of dopamine, it exhibits a new Explore regime denoting a random exploration of the space of action alternatives. The exploratory dynamics originates from the chaotic activity of the STN-GPe loop. When applied to the standard card choice experiment used in the imaging studies of Daw, O'Doherty, Dayan, Seymour, and Dolan (2006), the model favorably describes the exploratory behavior of human subjects.

Modeling the role of basal ganglia in saccade generation: is the indirect pathway the explorer?

We model the role played by the Basal Ganglia (BG) in the generation of voluntary saccadic eye movements. The BG model explicitly represents key nuclei like the striatum (caudate), Substantia Nigra pars reticulata (SNr) and compata (SNc), the Subthalamic Nucleus (STN), the two pallidal nuclei and Superior Colliculus. The model is cast within the Reinforcement Learning (RL) framework, with the dopamine representing the temporal difference error, the striatum serving as the critic, and the indirect pathway playing the role of the explorer. Performance of the model is evaluated on a set of tasks such as feature and conjunction searches, directional selectivity and a successive saccade task. Behavioral phenomena such as independence of search time on number of distractors in feature search and linear increase in search time with number of distractors in conjunction search are observed. It is also seen that saccadic reaction times are longer and search efficiency is impaired on diminished BG contribution, which corroborates with reported data obtained from Parkinson's Disease (PD) patients.

Informational dynamics of vasomotion in microvascular networks: a review.

Vasomotion refers to spontaneous oscillation of small vessels observed in many microvascular beds. It is an intrinsic phenomenon unrelated to cardiac rhythm or neural and hormonal regulation. Vasomotion is found to be particularly prominent under conditions of metabolic stress. In spite of a significant existent literature on vasomotion, its physiological and pathophysiological roles are not clear. It is thought that modulation of vasomotion by vasoactive substances released by metabolizing tissue plays a role in ensuring optimal delivery of nutrients to the tissue. Vasomotion rhythms exhibit a great variety of temporal patterns from regular oscillations to chaos. The nature of vasomotion rhythm is believed to be significant to its function, with chaotic vasomotion offering several physiological advantages over regular, periodic vasomotion. In this article, we emphasize that vasomotion is best understood as a network phenomenon. When there is a local metabolic demand in tissue, an ideal vascular response should extend beyond local microvasculature, with coordinated changes over multiple vascular segments. Mechanisms of information transfer over a vessel network have been discussed in the literature. The microvascular system may be regarded as a network of dynamic elements, interacting, either over the vascular anatomical network via gap junctions, or physiologically by exchange of vasoactive substances. Drawing analogies with spatiotemporal patterns in neuronal networks of central nervous system, we ask if properties like synchronization/desynchronization of vasomotors have special significance to microcirculation. Thus the contemporary literature throws up a novel view of microcirculation as a network that exhibits complex, spatiotemporal and informational dynamics.

Modeling basal ganglia for understanding Parkinsonian reaching movements.

We present a computational model that highlights the role of basal ganglia (BG) in generating simple reaching movements. The model is cast within the reinforcement learning (RL) framework with correspondence between RL components and neuroanatomy as follows: dopamine signal of substantia nigra pars compacta as the temporal difference error, striatum as the substrate for the critic, and the motor cortex as the actor. A key feature of this neurobiological interpretation is our hypothesis that the indirect pathway is the explorer. Chaotic activity, originating from the indirect pathway part of the model, drives the wandering, exploratory movements of the arm. Thus, the direct pathway subserves exploitation, while the indirect pathway subserves exploration. The motor cortex becomes more and more independent of the corrective influence of BG as training progresses. Reaching trajectories show diminishing variability with training. Reaching movements associated with Parkinson's disease (PD) are simulated by reducing dopamine and degrading the complexity of indirect pathway dynamics by switching it from chaotic to periodic behavior. Under the simulated PD conditions, the arm exhibits PD motor symptoms like tremor, bradykinesia and undershooting. The model echoes the notion that PD is a dynamical disease.

What do the basal ganglia do? A modeling perspective.

Basal ganglia (BG) constitute a network of seven deep brain nuclei involved in a variety of crucial brain functions including: action selection, action gating, reward based learning, motor preparation, timing, etc. In spite of the immense amount of data available today, researchers continue to wonder how a single deep brain circuit performs such a bewildering range of functions. Computational models of BG have focused on individual functions and fail to give an integrative picture of BG function. A major breakthrough in our understanding of BG function is perhaps the insight that activities of mesencephalic dopaminergic cells represent some form of 'reward' to the organism. This insight enabled application of tools from 'reinforcement learning,' a branch of machine learning, in the study of BG function. Nevertheless, in spite of these bright spots, we are far from the goal of arriving at a comprehensive understanding of these 'mysterious nuclei.' A comprehensive knowledge of BG function has the potential to radically alter treatment and management of a variety of BG-related neurological disorders (Parkinson's disease, Huntington's chorea, etc.) and neuropsychiatric disorders (schizophrenia, obsessive compulsive disorder, etc.) also. In this article, we review the existing modeling literature on BG and hypothesize an integrative picture of the function of these nuclei.

Bistable dynamics of cardiac cell models coupled by dynamic gap junctions linked to cardiac memory.

In an earlier study, we suggested that adaptive gap junctions (GJs) might be a basis of cardiac memory, a phenomenon which refers to persistent electrophysiological response of the heart to external pacing. Later, it was also shown that the proposed mechanism of adaptation of GJs is consistent with known electrophysiology of GJs. In the present article, we show that a pair of cardiac cell models coupled by dynamic, voltage-sensitive GJs exhibits bistable behavior under certain conditions. Three kinds of cell pairs are considered: (1) a Noble-Noble cell pair that represents adjacent cells in Purkinje network, (2) a pair of DiFranceso-Noble cells that represents adjacent SA nodal cells, and (3) a model of Noble cell coupled to Luo-Rudy cell model, which represents an interacting pair of a Purkinje fiber and a ventricular myocyte. Bistability is demonstrated in all the three cases. We suggest that this bistability might be an underlying factor behind cardiac memory. Focused analysis of a pair of Noble cell models showed that bistability is obtained only when the properties of GJs "match" with the properties of the pair of cells that is coupled by the GJs. This novel notion of match between GJs and cardiac cell types might give an insight into specialized distributions of various connexin proteins in cardiac tissue.

A model of indispensability of a large glial layer in cerebrovascular circulation.

We formulate the problem of oxygen delivery to neural tissue as a problem of association. Input to a pool of neurons in one brain area must be matched in space and time with metabolic inputs from the vascular network via the glial network. We thus have a model in which neural, glial, and vascular layers are connected bidirectionally, in that order. Connections between neuro-glial and glial-vascular stages are trained by an unsupervised learning mechanism such that input to the neural layer is sustained by the precisely patterned delivery of metabolic inputs from the vascular layer via the glial layer. Simulations show that the capacity of such a system to sustain patterns is weak when the glial layer is absent. Capacity is higher when a glial layer is present and increases with the layer size. The proposed formulation of neurovascular interactions raises many intriguing questions about the role of glial cells in cerebral circulation.

A phase dynamic model of systematic error in simple copying tasks.

A crucial insight into handwriting dynamics is embodied in the idea that stable, robust handwriting movements correspond to attractors of an oscillatory dynamical system. We present a phase dynamic model of visuomotor performance involved in copying simple oriented lines. Our studies on human performance in copying oriented lines revealed a systematic error pattern in orientation of drawn lines, i.e., lines at certain orientation are drawn more accurately than at other values. Furthermore, human subjects exhibit "flips" in direction at certain characteristic orientations. It is argued that this flipping behavior has its roots in the fact that copying process is inherently ambiguous-a line of given orientation may be drawn in two different (mutually opposite) directions producing the same end result. The systematic error patterns seen in human copying performance is probably a result of the attempt of our visuomotor system to cope with this ambiguity and still be able to produce accurate copying movements. The proposed nonlinear phase-dynamic model explains the experimentally observed copying error pattern and also the flipping behavior with remarkable accuracy.

A computational model of Parkinsonian handwriting that highlights the role of the indirect pathway in the basal ganglia.

Parkinsonian handwriting is typically characterized by micrographia, jagged line contour, and unusual fluctuations in pen velocity. In this paper we present a computational model of handwriting generation that highlights the role of the basal ganglia, particularly the indirect pathway. Whereas reduced dopamine levels resulted in reduced letter size, transition of STN-GPe dynamics from desynchronized (normal) to synchronized (PD) condition resulted in increased fluctuations in velocity in the model. We also present handwriting data from PD patients (n=34) who are at various stages of disease and had taken medication various lengths of time before the handwriting sessions. The patient data are compared with those of age-matched controls. PD handwriting statistically exhibited smaller size and larger velocity fluctuation compared to normal handwriting.

Desynchronized vasomotion and desynchronized fiber activation pattern enhance oxygenation in a model of skeletal muscle.

Although the full physiological significance of vasomotion is still debated, it is generally thought to have a role in optimizing tissue oxygenation parameters. We study the effect of vasomotion rhythm in skeletal muscle on oxygen transport using a computational model. The model is used: (i) to test a novel hypothesis that "vasomotors" form a chemical network in which the rhythm adapts to meet oxygen demand in skeletal muscle and (ii) to study the contribution of desynchronized/chaotic vasomotion in optimizing oxygen delivery to skeletal muscle. We formulate a 2D grid model of skeletal muscle consisting of an interleaved arrangement of vessels and muscle fibers fired by a motor neuronal network. The vasomotors too form a network interacting by chemical means. When positive (negative) synapses dominate, the neuronal network exhibits synchronized (desynchronized) activity. Similarly, when positive (negative) chemical interactions dominate, the vessels exhibit synchronized (desynchronized) activity. Optimal oxygenation is observed when both neuronal network and vasomotor network exhibit desynchronous activity. Muscle oxygenation is thought to result by interactions between the fiber/neuron network and the vessel network; optimal oxygenation depends on precise rhythm-related conditions on the two networks. The model provides interesting insights into the phenomenon of muscle fatigue.

Interpreting voltage-sensitivity of gap junctions as a mechanism of cardiac memory.

Memory in the nervous system is essentially a network effect, resulting from activity-dependent synaptic modification in a network of neurons. Like the nervous system, the heart is a network of cardiac cells electrically coupled by gap junctions. The heart too has memory, termed cardiac memory, whereby the effect of an external electrical activation persists long after the presentation of stimulus is terminated. We have earlier proposed that adaptation of gap junctions, as a function of membrane voltages of the cells that are coupled by the gap junctions, is related to cardiac memory [V.S. Chakravarthy, J. Ghosh, On Hebbian-like adaption in heart muscle: a proposal for "Cardiac Memory", Biol. Cybern. 76 (1997) 207, J. Krishnan, V.S. Chakravarthy, S. Radhakrishnan, On the role of gap junctions on cardiac memory effect, Comput. Cardiol. 32 (2005) 13]. Using the proposed mechanism, we demonstrate memory effect using computational models of interacting cell pairs. In this paper, we address the biological validity of the proposed mechanism of gap junctional adaptation. It is known from electrophysiology of gap junctions that the conductance of these channels adapts as a function of junctional voltage. At a first sight, this form of voltage dependence seems to be at variance with the form required by our mechanism. But we show, with the help of a theoretical model, that the proposed mechanism of voltage-dependent adaptation of gap junctions, is compatible with the known voltage-sensitivity of gap junctions observed in electrophysiological studies. Our analysis suggests a new significance of the voltage-sensitivity of gap junctions and its possible link to the phenomenon of cardiac memory.

An oscillator theory of motor unit recruitment.

The phenomenon of systematic recruitment of motor units with increasing demand load is usually explained by the size principle. Though this principle successfully explains the gain-related aspects of muscle force generation, it does not address the need for desynchronization of motor unit activities in order to produce a smooth tension profile at the level of whole muscle, while individual muscle fibers are "twitching." We propose an oscillator model of motor neurons in which a pool of motor neurons fires a bundle of muscle fibers. Although individual muscle fibers have a complicated tension profile, the tension produced by the entire bundle is regulated and follows a command signal accurately. This is shown to be possible because of uncorrelated activity produced by local inhibitory connections among motor neurons. Connections that produce synchronized oscillations result in uncontrolled contractions of the muscle. These results seem to suggest that while synchronized activity indicates pathology and disease, desynchronized activity is the precondition for normal muscle function. Physiological evidence for the proposed theory of motor unit synchronization is presented.

A computational model that links non-periodic vasomotion to enhanced oxygenation in skeletal muscle.

We propose a model of a capillary network in which chaotic capillary activity is crucial for efficient oxygenation of a muscle fiber. Tissue oxygenation by microcirculation is controlled by a complex pattern of opening and closing of precapillary sphincters, a phenomenon known as vasomotion. We model the individual precapillary sphincter as a non-linear oscillator that exhibits perfectly periodic vasomotion in isolation. The precapillary sphincters surrounding an active fiber are considered as a network; specific modes of interaction within this network result in complex patterns of vasomotion. In our model, efficient oxygenation of the fiber depends crucially on the mode of interaction among the vasomotions of the individual capillaries. Network interactions that lead to chaotic vasomotion are found to be essential for meeting the tissue oxygen demands precisely. Interactions that cause regular rhythmic patterns of vasomotion fail to meet oxygenation demands accurately.

Effect of chaotic vasomotion in skeletal muscle on tissue oxygenation.

Vasomotion refers to spontaneous variations in the lumen size of small vessels, with a plausible role in regulation of various aspects of microcirculation. We propose a computational model of vasomotion in skeletal muscle in which the pattern of vasomotion is shown to critically determine the efficiency of oxygenation of a muscle fiber. In this model, precapillary sphincters are modeled as nonlinear oscillators. We hypothesize that these sphincters interact via exchange of vasoactive substances. As a consequence, vasomotion is described as a phenomenon associated with a network of nonlinear oscillators. As a specific instance, we model the vasomotion of precapillary sphincters surrounding an active fiber. The sphincters coordinate their rhythms so as to minimize oxygen deficit in the fiber. Our modeling studies indicate that efficient oxygenation of the fiber depends crucially on the mode of interaction among the vasomotions of individual sphincters. While chaotic forms of vasomotion enhanced oxygenation, regular patterns of vasomotion failed to meet the oxygenation demand accurately.

The role of the basal ganglia in exploration in a neural model based on reinforcement learning.

We present a computational model of basal ganglia as a key player in exploratory behavior. The model describes exploration of a virtual rat in a simulated water pool experiment. The virtual rat is trained using a reward-based or reinforcement learning paradigm which requires units with stochastic behavior for exploration of the system's state space. We model the Subthalamic Nucleus-Globus Pallidus externa (STN-GPe) segment of the basal ganglia as a pair of neuronal layers with oscillatory dynamics, exhibiting a variety of dynamic regimes such as chaos, traveling waves and clustering. Invoking the property of chaotic systems to explore state-space, we suggest that the complex exploratory dynamics of STN-GPe system in conjunction with dopamine-based reward signaling from the Substantia Nigra pars compacta (SNc) present the two key ingredients of a reinforcement learning system.