Correction: The contribution of the basal ganglia and cerebellum to motor learning: A neuro-computational approach.

[This corrects the article DOI: 10.1371/journal.pcbi.1011024.].

Exploration behavior after reversals is predicted by STN-GPe synaptic plasticity in a basal ganglia model.

Humans can quickly adapt their behavior to changes in the environment. Classical reversal learning tasks mainly measure how well participants can disengage from a previously successful behavior but not how alternative responses are explored. Here, we propose a novel 5-choice reversal learning task with alternating position-reward contingencies to study exploration behavior after a reversal. We compare human exploratory saccade behavior with a prediction obtained from a neuro-computational model of the basal ganglia. A new synaptic plasticity rule for learning the connectivity between the subthalamic nucleus (STN) and external globus pallidus (GPe) results in exploration biases to previously rewarded positions. The model simulations and human data both show that during experimental experience exploration becomes limited to only those positions that have been rewarded in the past. Our study demonstrates how quite complex behavior may result from a simple sub-circuit within the basal ganglia pathways.

The contribution of the basal ganglia and cerebellum to motor learning: A neuro-computational approach.

Motor learning involves a widespread brain network including the basal ganglia, cerebellum, motor cortex, and brainstem. Despite its importance, little is known about how this network learns motor tasks and which role different parts of this network take. We designed a systems-level computational model of motor learning, including a cortex-basal ganglia motor loop and the cerebellum that both determine the response of central pattern generators in the brainstem. First, we demonstrate its ability to learn arm movements toward different motor goals. Second, we test the model in a motor adaptation task with cognitive control, where the model replicates human data. We conclude that the cortex-basal ganglia loop learns via a novelty-based motor prediction error to determine concrete actions given a desired outcome, and that the cerebellum minimizes the remaining aiming error.

Virtual deep brain stimulation: Multiscale co-simulation of a spiking basal ganglia model and a whole-brain mean-field model with The Virtual Brain.

Deep brain stimulation (DBS) has been successfully applied in various neurodegenerative diseases as an effective symptomatic treatment. However, its mechanisms of action within the brain network are still poorly understood. Many virtual DBS models analyze a subnetwork around the basal ganglia and its dynamics as a spiking network with their details validated by experimental data. However, connectomic evidence shows widespread effects of DBS affecting many different cortical and subcortical areas. From a clinical perspective, various effects of DBS besides the motoric impact have been demonstrated. The neuroinformatics platform The Virtual Brain (TVB) offers a modeling framework allowing us to virtually perform stimulation, including DBS, and forecast the outcome from a dynamic systems perspective prior to invasive surgery with DBS lead placement. For an accurate prediction of the effects of DBS, we implement a detailed spiking model of the basal ganglia, which we combine with TVB via our previously developed co-simulation environment. This multiscale co-simulation approach builds on the extensive previous literature of spiking models of the basal ganglia while simultaneously offering a whole-brain perspective on widespread effects of the stimulation going beyond the motor circuit. In the first demonstration of our model, we show that virtual DBS can move the firing rates of a Parkinson's disease patient's thalamus - basal ganglia network towards the healthy regime while, at the same time, altering the activity in distributed cortical regions with a pronounced effect in frontal regions. Thus, we provide proof of concept for virtual DBS in a co-simulation environment with TVB. The developed modeling approach has the potential to optimize DBS lead placement and configuration and forecast the success of DBS treatment for individual patients.

BOLD Monitoring in the Neural Simulator ANNarchy.

Multi-scale network models that simultaneously simulate different measurable signals at different spatial and temporal scales, such as membrane potentials of single neurons, population firing rates, local field potentials, and blood-oxygen-level-dependent (BOLD) signals, are becoming increasingly popular in computational neuroscience. The transformation of the underlying simulated neuronal activity of these models to simulated non-invasive measurements, such as BOLD signals, is particularly relevant. The present work describes the implementation of a BOLD monitor within the neural simulator ANNarchy to allow an on-line computation of simulated BOLD signals from neural network models. An active research topic regarding the simulation of BOLD signals is the coupling of neural processes to cerebral blood flow (CBF) and cerebral metabolic rate of oxygen (CMRO2). The flexibility of ANNarchy allows users to define this coupling with a high degree of freedom and thus, not only allows to relate mesoscopic network models of populations of spiking neurons to experimental BOLD data, but also to investigate different hypotheses regarding the coupling between neural processes, CBF and CMRO2 with these models. In this study, we demonstrate how simulated BOLD signals can be obtained from a network model consisting of multiple spiking neuron populations. We first demonstrate the use of the Balloon model, the predominant model for simulating BOLD signals, as well as the possibility of using novel user-defined models, such as a variant of the Balloon model with separately driven CBF and CMRO2 signals. We emphasize how different hypotheses about the coupling between neural processes, CBF and CMRO2 can be implemented and how these different couplings affect the simulated BOLD signals. With the BOLD monitor presented here, ANNarchy provides a tool for modelers who want to relate their network models to experimental MRI data and for scientists who want to extend their studies of the coupling between neural processes and the BOLD signal by using modeling approaches. This facilitates the investigation and model-based analysis of experimental BOLD data and thus improves multi-scale understanding of neural processes in humans.

Enhanced habit formation in Tourette patients explained by shortcut modulation in a hierarchical cortico-basal ganglia model.

Devaluation protocols reveal that Tourette patients show an increased propensity to habitual behaviors as they continue to respond to devalued outcomes in a cognitive stimulus-response-outcome association task. We use a neuro-computational model of hierarchically organized cortico-basal ganglia-thalamo-cortical loops to shed more light on habit formation and its alteration in Tourette patients. In our model, habitual behavior emerges from cortico-thalamic shortcut connections, where enhanced habit formation can be linked to faster plasticity in the shortcut or to a stronger feedback from the shortcut to the basal ganglia. We explore two major hypotheses of Tourette pathophysiology-local striatal disinhibition and increased dopaminergic modulation of striatal medium spiny neurons-as causes for altered shortcut activation. Both model changes altered shortcut functioning and resulted in higher rates of responses towards devalued outcomes, similar to what is observed in Tourette patients. We recommend future experimental neuroscientific studies to locate shortcuts between cortico-basal ganglia-thalamo-cortical loops in the human brain and study their potential role in health and disease.

A spiking model of basal ganglia dynamics in stopping behavior supported by arkypallidal neurons.

The common view that stopping action plans by the basal ganglia is achieved mainly by the subthalamic nucleus alone due to its direct excitatory projection onto the output nuclei of the basal ganglia has been challenged by recent findings. The proposed "pause-then-cancel" model suggests that the subthalamic nucleus provides a rapid stimulus-unspecific "pause" signal, followed by a stop-cue-specific "cancel" signal from striatum-projecting arkypallidal neurons. To determine more precisely the relative contribution of the different basal ganglia nuclei in stopping, we simulated a stop-signal task with a spiking neuron model of the basal ganglia, considering recently discovered connections from the arkypallidal neurons, and cortex-projecting GPe neurons. For the arkypallidal and prototypical GPe neurons, we obtained neuron model parameters by fitting their neuronal responses to published experimental data. Our model replicates findings of stop-signal tasks at neuronal and behavioral levels. We provide evidence for the existence of a stop-related cortical input to the arkypallidal and cortex-projecting GPe neurons such that the stop responses of the subthalamic nucleus, the arkypallidal neurons, and the cortex-projecting GPe neurons complement each other to achieve functional stopping behavior. Particularly, the cortex-projecting GPe neurons may complement the stopping within the basal ganglia caused by the arkypallidal and STN neurons by diminishing cortical go-related processes. Furthermore, we predict effects of lesions on stopping performance and propose that arkypallidal neurons mainly participate in stopping by inhibiting striatal neurons of the indirect rather than the direct pathway.

A computational model-based analysis of basal ganglia pathway changes in Parkinson's disease inferred from resting-state fMRI.

Previous computational model-based approaches for understanding the dynamic changes related to Parkinson's disease made particular assumptions about Parkinson's disease-related activity changes or specified dopamine-dependent activation or learning rules. Inspired by recent model-based analysis of resting-state fMRI, we have taken a data-driven approach. We fit the free parameters of a spiking neuro-computational model to match correlations of blood oxygen level-dependent signals between different basal ganglia nuclei and obtain subject-specific neuro-computational models of two subject groups: Parkinson patients and matched controls. When comparing mean firing rates at rest and connectivity strengths between the control and Parkinsonian model groups, several significant differences were found that are consistent with previous experimental observations. We discuss the implications of our approach and compare its results also with the popular "rate model" of the basal ganglia. Our study suggests that a model-based analysis of imaging data from healthy and Parkinsonian subjects is a promising approach for the future to better understand Parkinson-related changes in the basal ganglia and corresponding treatments.

Habit learning in hierarchical cortex-basal ganglia loops.

How do the multiple cortico-basal ganglia-thalamo-cortical loops interact? Are they parallel and fully independent or controlled by an arbitrator, or are they hierarchically organized? We introduce here a set of four key concepts, integrated and evaluated by means of a neuro-computational model, that bring together current ideas regarding cortex-basal ganglia interactions in the context of habit learning. According to key concept 1, each loop learns to select an intermediate objective at a different abstraction level, moving from goals in the ventral striatum to motor in the putamen. Key concept 2 proposes that the cortex integrates the basal ganglia selection with environmental information regarding the achieved objective. Key concept 3 claims shortcuts between loops, and key concept 4 predicts that loops compute their own prediction error signal for learning. Computational benefits of the key concepts are demonstrated. Contrasting with former concepts of habit learning, the loops collaborate to select goal-directed actions while training slower shortcuts develops habitual responses.

The subthalamic nucleus-external globus pallidus loop biases exploratory decisions towards known alternatives: a neuro-computational study.

Theories and models of the basal ganglia have mainly focused on the role of three different corticothalamic pathways: direct, indirect and hyperdirect. Although the indirect and the hyperdirect pathways are linked through the bidirectional connections between the subthalamic nucleus (STN) and the external globus pallidus (GPe), the role of their interactions has been mainly discussed in the context of a dysfunction (abnormal oscillations in Parkinson's disease) and not of its function. We here propose a novel role for the loop formed by the STN and the GPe. We show, through a neuro-computational model, that this loop can bias the selection of actions during the exploratory period after a change in the environmental conditions towards alternative responses. Testing well-known alternative solutions before completely random actions can reduce the time required for the search of a new response after a rule change. Our simulations further show that the knowledge acquired by the indirect pathway can be transferred into a stable memory via learning in the hyperdirect pathway to establish the blocking of unwanted responses. After a rule switch, first the indirect pathway learns to inhibit the previously correct actions. Once the new correct association is learned, the inhibition is transferred to the hyperdirect pathway through synaptic plasticity.

On the Role of Cortex-Basal Ganglia Interactions for Category Learning: A Neurocomputational Approach.

In addition to the prefrontal cortex (PFC), the basal ganglia (BG) have been increasingly often reported to play a fundamental role in category learning, but the circuit mechanisms mediating their interaction remain to be explored. We developed a novel neurocomputational model of category learning that particularly addresses the BG-PFC interplay. We propose that the BG bias PFC activity by removing the inhibition of cortico-thalamo-cortical loop and thereby provide a teaching signal to guide the acquisition of category representations in the corticocortical associations to the PFC. Our model replicates key behavioral and physiological data of macaque monkey learning a prototype distortion task from Antzoulatos and Miller (2011) Our simulations allowed us to gain a deeper insight into the observed drop of category selectivity in striatal neurons seen in the experimental data and in the model. The simulation results and a new analysis of the experimental data based on the model's predictions show that the drop in category selectivity of the striatum emerges as the variability of responses in the striatum rises when confronting the BG with an increasingly larger number of stimuli to be classified. The neurocomputational model therefore provides new testable insights of systems-level brain circuits involved in category learning that may also be generalized to better understand other cortico-BG-cortical loops.SIGNIFICANCE STATEMENT Inspired by the idea that basal ganglia (BG) teach the prefrontal cortex (PFC) to acquire category representations, we developed a novel neurocomputational model and tested it on a task that was recently applied in monkey experiments. As an advantage over previous models of category learning, our model allows to compare simulation data with single-cell recordings in PFC and BG. We not only derived model predictions, but already verified a prediction to explain the observed drop in striatal category selectivity. When testing our model with a simple, real-world face categorization task, we observed that the fast striatal learning with a performance of 85% correct responses can teach the slower PFC learning to push the model performance up to almost 100%.

A spiking neural network based on the basal ganglia functional anatomy.

We introduce a spiking neural network of the basal ganglia capable of learning stimulus-action associations. We model learning in the three major basal ganglia pathways, direct, indirect and hyperdirect, by spike time dependent learning and considering the amount of dopamine available (reward). Moreover, we allow to learn a cortico-thalamic pathway that bypasses the basal ganglia. As a result the system develops new functionalities for the different basal ganglia pathways: The direct pathway selects actions by disinhibiting the thalamus, the hyperdirect one suppresses alternatives and the indirect pathway learns to inhibit common mistakes. Numerical experiments show that the system is capable of learning sets of either deterministic or stochastic rules.

Mean-field description and propagation of chaos in networks of Hodgkin-Huxley and FitzHugh-Nagumo neurons.

We derive the mean-field equations arising as the limit of a network of interacting spiking neurons, as the number of neurons goes to infinity. The neurons belong to a fixed number of populations and are represented either by the Hodgkin-Huxley model or by one of its simplified version, the FitzHugh-Nagumo model. The synapses between neurons are either electrical or chemical. The network is assumed to be fully connected. The maximum conductances vary randomly. Under the condition that all neurons' initial conditions are drawn independently from the same law that depends only on the population they belong to, we prove that a propagation of chaos phenomenon takes place, namely that in the mean-field limit, any finite number of neurons become independent and, within each population, have the same probability distribution. This probability distribution is a solution of a set of implicit equations, either nonlinear stochastic differential equations resembling the McKean-Vlasov equations or non-local partial differential equations resembling the McKean-Vlasov-Fokker-Planck equations. We prove the well-posedness of the McKean-Vlasov equations, i.e. the existence and uniqueness of a solution. We also show the results of some numerical experiments that indicate that the mean-field equations are a good representation of the mean activity of a finite size network, even for modest sizes. These experiments also indicate that the McKean-Vlasov-Fokker-Planck equations may be a good way to understand the mean-field dynamics through, e.g. a bifurcation analysis.Mathematics Subject Classification (2000): 60F99, 60B10, 92B20, 82C32, 82C80, 35Q80.