Conditions for Synaptic Specificity during the Maintenance Phase of Synaptic Plasticity.

Activity-dependent modifications of synaptic efficacies are a cellular substrate of learning and memory. Experimental evidence shows that these modifications are synapse specific and that the long-lasting effects are associated with the sustained increase in concentration of specific proteins like PKMζ However, such proteins are likely to diffuse away from their initial synaptic location and spread out to neighboring synapses, potentially compromising synapse specificity. In this article, we address the issue of synapse specificity during memory maintenance. Assuming that the long-term maintenance of synaptic plasticity is accomplished by a molecular switch, we carry out analytical calculations and perform simulations using the reaction-diffusion package in NEURON to determine the limits of synapse specificity during maintenance. Moreover, we explore the effects of the diffusion and degradation rates of proteins and of the geometrical characteristics of dendritic spines on synapse specificity. We conclude that the necessary conditions for synaptic specificity during maintenance require that molecular switches reside in dendritic spines. The requirement for synaptic specificity when the molecular switch resides in spines still imposes strong limits on the diffusion and turnover of rates of maintenance molecules, as well as on the morphologic properties of synaptic spines. These constraints are quite general and apply to most existing models suggested for maintenance. The parameter values can be experimentally evaluated, and if they do not fit the appropriate predicted range, the validity of this class of maintenance models would be challenged.

The Role of Multiple Neuromodulators in Reinforcement Learning That Is Based on Competition between Eligibility Traces.

The ability to maximize reward and avoid punishment is essential for animal survival. Reinforcement learning (RL) refers to the algorithms used by biological or artificial systems to learn how to maximize reward or avoid negative outcomes based on past experiences. While RL is also important in machine learning, the types of mechanistic constraints encountered by biological machinery might be different than those for artificial systems. Two major problems encountered by RL are how to relate a stimulus with a reinforcing signal that is delayed in time (temporal credit assignment), and how to stop learning once the target behaviors are attained (stopping rule). To address the first problem synaptic eligibility traces were introduced, bridging the temporal gap between a stimulus and its reward. Although, these were mere theoretical constructs, recent experiments have provided evidence of their existence. These experiments also reveal that the presence of specific neuromodulators converts the traces into changes in synaptic efficacy. A mechanistic implementation of the stopping rule usually assumes the inhibition of the reward nucleus; however, recent experimental results have shown that learning terminates at the appropriate network state even in setups where the reward nucleus cannot be inhibited. In an effort to describe a learning rule that solves the temporal credit assignment problem and implements a biologically plausible stopping rule, we proposed a model based on two separate synaptic eligibility traces, one for long-term potentiation (LTP) and one for long-term depression (LTD), each obeying different dynamics and having different effective magnitudes. The model has been shown to successfully generate stable learning in recurrent networks. Although, the model assumes the presence of a single neuromodulator, evidence indicates that there are different neuromodulators for expressing the different traces. What could be the role of different neuromodulators for expressing the LTP and LTD traces? Here we expand on our previous model to include several neuromodulators, and illustrate through various examples how different these contribute to learning reward-timing within a wide set of training paradigms and propose further roles that multiple neuromodulators can play in encoding additional information of the rewarding signal.

A Simple Network Architecture Accounts for Diverse Reward Time Responses in Primary Visual Cortex.

Many actions performed by animals and humans depend on an ability to learn, estimate, and produce temporal intervals of behavioral relevance. Exemplifying such learning of cued expectancies is the observation of reward-timing activity in the primary visual cortex (V1) of rodents, wherein neural responses to visual cues come to predict the time of future reward as behaviorally experienced in the past. These reward-timing responses exhibit significant heterogeneity in at least three qualitatively distinct classes: sustained increase or sustained decrease in firing rate until the time of expected reward, and a class of cells that reach a peak in firing at the expected delay. We elaborate upon our existing model by including inhibitory and excitatory units while imposing simple connectivity rules to demonstrate what role these inhibitory elements and the simple architectures play in sculpting the response dynamics of the network. We find that simply adding inhibition is not sufficient for obtaining the different distinct response classes, and that a broad distribution of inhibitory projections is necessary for obtaining peak-type responses. Furthermore, although changes in connection strength that modulate the effects of inhibition onto excitatory units have a strong impact on the firing rate profile of these peaked responses, the network exhibits robustness in its overall ability to predict the expected time of reward. Finally, we demonstrate how the magnitude of expected reward can be encoded at the expected delay in the network and how peaked responses express this reward expectancy. SIGNIFICANCE STATEMENT: Heterogeneity in single-neuron responses is a common feature of neuronal systems, although sometimes, in theoretical approaches, it is treated as a nuisance and seldom considered as conveying a different aspect of a signal. In this study, we focus on the heterogeneous responses in the primary visual cortex of rodents trained with a predictable delayed reward time. We describe under what conditions this heterogeneity can arise by self-organization, and what information it can convey. This study, while focusing on a specific system, provides insight onto how heterogeneity can arise in general while also shedding light onto mechanisms of reinforcement learning using realistic biological assumptions.

Visually cued action timing in the primary visual cortex.

Most behaviors are generated in three steps: sensing the external world, processing that information to instruct decision-making, and producing a motor action. Sensory areas, especially primary sensory cortices, have long been held to be involved only in the first step of this sequence. Here, we develop a visually cued interval timing task that requires rats to decide when to perform an action following a brief visual stimulus. Using single-unit recordings and optogenetics in this task, we show that activity generated by the primary visual cortex (V1) embodies the target interval and may instruct the decision to time the action on a trial-by-trial basis. A spiking neuronal model of local recurrent connections in V1 produces neural responses that predict and drive the timing of future actions, rationalizing our observations. Our data demonstrate that the primary visual cortex may contribute to the instruction of visually cued timed actions.

Pacemaker rate and depolarization block in nigral dopamine neurons: a somatic sodium channel balancing act.

Midbrain dopamine (DA) neurons are slow intrinsic pacemakers that undergo depolarization (DP) block upon moderate stimulation. Understanding DP block is important because it has been correlated with the clinical efficacy of chronic antipsychotic drug treatment. Here we describe how voltage-gated sodium (Na(V)) channels regulate DP block and pacemaker activity in DA neurons of the substantia nigra using rat brain slices. The distribution, density, and gating of Na(V) currents were manipulated by blocking native channels with tetrodotoxin and by creating virtual channels and anti-channels with dynamic clamp. Although action potentials initiate in the axon initial segment and Na(V) channels are distributed in multiple dendrites, selective reduction of Na(V) channel activity in the soma was sufficient to decrease pacemaker frequency and increase susceptibility to DP block. Conversely, increasing somatic Na(V) current density raised pacemaker frequency and lowered susceptibility to DP block. Finally, when Na(V) currents were restricted to the soma, pacemaker activity occurred at abnormally high rates due to excessive local subthreshold Na(V) current. Together with computational simulations, these data show that both the slow pacemaker rate and the sensitivity to DP block that characterizes DA neurons result from the low density of somatic Na(V) channels. More generally, we conclude that the somatodendritic distribution of Na(V) channels is a major determinant of repetitive spiking frequency.

Functional characterization of ether-à-go-go-related gene potassium channels in midbrain dopamine neurons - implications for a role in depolarization block.

Bursting activity by midbrain dopamine neurons reflects the complex interplay between their intrinsic pacemaker activity and synaptic inputs. Although the precise mechanism responsible for the generation and modulation of bursting in vivo has yet to be established, several ion channels have been implicated in the process. Previous studies with nonselective blockers suggested that ether-à-go-go-related gene (ERG) K(+) channels are functionally significant. Here, electrophysiology with selective chemical and peptide ERG channel blockers (E-4031 and rBeKm-1) and computational methods were used to define the contribution made by ERG channels to the firing properties of midbrain dopamine neurons in vivo and in vitro. Selective ERG channel blockade increased the frequency of spontaneous activity as well as the response to depolarizing current pulses without altering spike frequency adaptation. ERG channel block also accelerated entry into depolarization inactivation during bursts elicited by virtual NMDA receptors generated with the dynamic clamp, and significantly prolonged the duration of the sustained depolarization inactivation that followed pharmacologically evoked bursts. In vivo, somatic ERG blockade was associated with an increase in bursting activity attributed to a reduction in doublet firing. Taken together, these results show that dopamine neuron ERG K(+) channels play a prominent role in limiting excitability and in minimizing depolarization inactivation. As the therapeutic actions of antipsychotic drugs are associated with depolarization inactivation of dopamine neurons and blockade of cardiac ERG channels is a prominent side effect of these drugs, ERG channels in the central nervous system may represent a novel target for antipsychotic drug development.

Ca2+ alternans in a cardiac myocyte model that uses moment equations to represent heterogeneous junctional SR Ca2+.

Multiscale whole-cell models that accurately represent local control of Ca2+-induced Ca2+ release in cardiac myocytes can reproduce high-gain Ca2+ release that is graded with changes in membrane potential. Using a recently introduced formalism that represents heterogeneous local Ca2+ using moment equations, we present a model of cardiac myocyte Ca2+ cycling that exhibits alternating sarcoplasmic reticulum (SR) Ca2+ release when periodically stimulated by depolarizing voltage pulses. The model predicts that the distribution of junctional SR [Ca2+] across a large population of Ca2+ release units is distinct on alternating cycles. Load-release and release-uptake functions computed from this model give insight into how Ca2+ fluxes and stimulation frequency combine to determine the presence or absence of Ca2+ alternans. Our results show that the conditions for the onset of Ca2+ alternans cannot be explained solely by the steepness of the load-release function, but that changes in the release-uptake process also play an important role. We analyze the effect of the junctional SR refilling time constant on Ca2+ alternans and conclude that physiologically realistic models of defective Ca2+ cycling must represent the dynamics of heterogeneous junctional SR [Ca2+] without assuming rapid equilibration of junctional and network SR [Ca2+].

Regulation of firing frequency in a computational model of a midbrain dopaminergic neuron.

Dopaminergic (DA) neurons of the mammalian midbrain exhibit unusually low firing frequencies in vitro. Furthermore, injection of depolarizing current induces depolarization block before high frequencies are achieved. The maximum steady and transient rates are about 10 and 20 Hz, respectively, despite the ability of these neurons to generate bursts at higher frequencies in vivo. We use a three-compartment model calibrated to reproduce DA neuron responses to several pharmacological manipulations to uncover mechanisms of frequency limitation. The model exhibits a slow oscillatory potential (SOP) dependent on the interplay between the L-type Ca(2+) current and the small conductance K(+) (SK) current that is unmasked by fast Na(+) current block. Contrary to previous theoretical work, the SOP does not pace the steady spiking frequency in our model. The main currents that determine the spontaneous firing frequency are the subthreshold L-type Ca(2+) and the A-type K(+) currents. The model identifies the channel densities for the fast Na(+) and the delayed rectifier K(+) currents as critical parameters limiting the maximal steady frequency evoked by a depolarizing pulse. We hypothesize that the low maximal steady frequencies result from a low safety factor for action potential generation. In the model, the rate of Ca(2+) accumulation in the distal dendrites controls the transient initial frequency in response to a depolarizing pulse. Similar results are obtained when the same model parameters are used in a multi-compartmental model with a realistic reconstructed morphology, indicating that the salient contributions of the dendritic architecture have been captured by the simpler model.

Moment closure for local control models of calcium-induced calcium release in cardiac myocytes.

In prior work, we introduced a probability density approach to modeling local control of Ca2+-induced Ca2+ release in cardiac myocytes, where we derived coupled advection-reaction equations for the time-dependent bivariate probability density of subsarcolemmal subspace and junctional sarcoplasmic reticulum (SR) [Ca2+] conditioned on Ca2+ release unit (CaRU) state. When coupled to ordinary differential equations (ODEs) for the bulk myoplasmic and network SR [Ca2+], a realistic but minimal model of cardiac excitation-contraction coupling was produced that avoids the computationally demanding task of resolving spatial aspects of global Ca2+ signaling, while accurately representing heterogeneous local Ca2+ signals in a population of diadic subspaces and junctional SR depletion domains. Here we introduce a computationally efficient method for simulating such whole cell models when the dynamics of subspace [Ca2+] are much faster than those of junctional SR [Ca2+]. The method begins with the derivation of a system of ODEs describing the time-evolution of the moments of the univariate probability density functions for junctional SR [Ca2+] jointly distributed with CaRU state. This open system of ODEs is then closed using an algebraic relationship that expresses the third moment of junctional SR [Ca2+] in terms of the first and second moments. In simulated voltage-clamp protocols using 12-state CaRUs that respond to the dynamics of both subspace and junctional SR [Ca2+], this moment-closure approach to simulating local control of excitation-contraction coupling produces high-gain Ca2+ release that is graded with changes in membrane potential, a phenomenon not exhibited by common pool models. Benchmark simulations indicate that the moment-closure approach is nearly 10,000-times more computationally efficient than corresponding Monte Carlo simulations while leading to nearly identical results. We conclude by applying the moment-closure approach to study the restitution of Ca2+-induced Ca2+ release during simulated two-pulse voltage-clamp protocols.

The dynamics of luminal depletion and the stochastic gating of Ca2+-activated Ca2+ channels and release sites.

Single channel models of intracellular calcium (Ca(2+)) channels such as the 1,4,5-trisphosphate receptor and ryanodine receptor often assume that Ca(2+)-dependent transitions are mediated by constant background cytosolic [Ca(2+)]. This assumption neglects the fact that Ca(2+) released by open channels may influence subsequent gating through the processes of Ca(2+)-activation or inactivation. Similarly, the influence of the dynamics of luminal depletion on the stochastic gating of intracellular Ca(2+) channels is often neglected, in spite of the fact that the sarco/endoplasmic reticulum [Ca(2+)] near the luminal face of intracellular Ca(2+) channels influences the driving force for Ca(2+), the rate of Ca(2+) release, and the magnitude and time course of the consequent increase in cytosolic domain [Ca(2+)]. Here we analyze how the steady-state open probability of several minimal Ca(2+)-regulated Ca(2+) channel models depends on the conductance of the channel and the time constants for the relaxation of elevated cytosolic [Ca(2+)] and depleted luminal [Ca(2+)] to the bulk [Ca(2+)] of both compartments. Our approach includes Monte Carlo simulation as well as numerical solution of a system of advection-reaction equations for the multivariate probability density of elevated cytosolic [Ca(2+)] and depleted luminal [Ca(2+)] conditioned on each state of the stochastically gating channel. Both methods are subsequently used to study the role of luminal depletion in the dynamics of Ca(2+) puff/spark termination in release sites composed of Ca(2+) channels that are activated, but not inactivated, by cytosolic Ca(2+). The probability density approach shows that such minimal Ca(2+) release site models may exhibit puff/spark-like dynamics in either of two distinct parameter regimes. In one case, puffs/spark termination is due to the process of stochastic attrition and facilitated by rapid Ca(2+) domain collapse [cf. DeRemigio, H., Smith, G., 2005. The dynamics of stochastic attrition viewed as an absorption time on a terminating Markov chain. Cell Calcium 38, 73-86]. In the second case, puff/spark termination is promoted by the local depletion of luminal Ca(2+).

A probability density approach to modeling local control of calcium-induced calcium release in cardiac myocytes.

We present a probability density approach to modeling localized Ca2+ influx via L-type Ca2+ channels and Ca2+-induced Ca2+ release mediated by clusters of ryanodine receptors during excitation-contraction coupling in cardiac myocytes. Coupled advection-reaction equations are derived relating the time-dependent probability density of subsarcolemmal subspace and junctional sarcoplasmic reticulum [Ca2+] conditioned on "Ca2+ release unit" state. When these equations are solved numerically using a high-resolution finite difference scheme and the resulting probability densities are coupled to ordinary differential equations for the bulk myoplasmic and sarcoplasmic reticulum [Ca2+], a realistic but minimal model of cardiac excitation-contraction coupling is produced. Modeling Ca2+ release unit activity using this probability density approach avoids the computationally demanding task of resolving spatial aspects of global Ca2+ signaling, while accurately representing heterogeneous local Ca2+ signals in a population of diadic subspaces and junctional sarcoplasmic reticulum depletion domains. The probability density approach is validated for a physiologically realistic number of Ca2+ release units and benchmarked for computational efficiency by comparison to traditional Monte Carlo simulations. In simulated voltage-clamp protocols, both the probability density and Monte Carlo approaches to modeling local control of excitation-contraction coupling produce high-gain Ca2+ release that is graded with changes in membrane potential, a phenomenon not exhibited by so-called "common pool" models. However, a probability density calculation can be significantly faster than the corresponding Monte Carlo simulation, especially when cellular parameters are such that diadic subspace [Ca2+] is in quasistatic equilibrium with junctional sarcoplasmic reticulum [Ca2+] and, consequently, univariate rather than multivariate probability densities may be employed.

A multivariate population density model of the dLGN/PGN relay.

Using a population density approach we study the dynamics of two interacting collections of integrate-and-fire-or-burst (IFB) neurons representing thalamocortical (TC) cells from the dorsal lateral geniculate nucleus (dLGN) and thalamic reticular (RE) cells from the perigeniculate nucleus (PGN). Each population of neurons is described by a multivariate probability density function that satisfies a conservation equation with appropriately defined probability fluxes and boundary conditions. The state variables of each neuron are the membrane potential and the inactivation gating variable of the low-threshold Ca2+ current I(T). The synaptic coupling of the populations and external excitatory drive are modeled by instantaneous jumps in the membrane potential of postsynaptic neurons. The population density model is validated by comparing its response to time-varying retinal input to Monte Carlo simulations of the corresponding IFB network composed of 100 to 1,000 cells per population. In the absence of retinal input, the population density model exhibits rhythmic bursting similar to the 7 to 14 Hz oscillations associated with slow wave sleep that require feedback inhibition from RE to TC cells. When the TC and RE cell potassium leakage conductances are adjusted to represent cholingergic neuromodulation and arousal of the network, rhythmic bursting of the probability density model may either persists or be eliminated depending on the number of excitatory (TC to RE) or inhibitory (RE to TC) connections made by each presynaptic cell. When the probability density model is stimulated with constant retinal input (10-100 spikes/sec), a wide range of responses are observed depending on cellular parameters and network connectivity. These include asynchronous burst and tonic spikes, sleep spindle-like rhythmic bursting, and oscillations in population firing rate that are distinguishable from sleep spindles due to their amplitude, frequency, or the presence of tonic spikes. In this context of dLGN/PGN network modeling, we find the population density approach using 2,500 mesh points and resolving membrane voltage to 0.7 mV is over 30 times more efficient than 1,000-cell Monte Carlo simulations.

Feedback inhibition and throughput properties of an integrate-and-fire-or-burst network model of retinogeniculate transmission.

Computational modeling has played an important role in the dissection of the biophysical basis of rhythmic oscillations in thalamus that are associated with sleep and certain forms of epilepsy. In contrast, the dynamic filter properties of thalamic relay nuclei during states of arousal are not well understood. Here we present a modeling and simulation study of the throughput properties of the visually driven dorsal lateral geniculate nucleus (dLGN) in the presence of feedback inhibition from the perigeniculate nucleus (PGN). We employ thalamocortical (TC) and thalamic reticular (RE) versions of a minimal integrate-and-fire-or-burst type model and a one-dimensional, two-layered network architecture. Potassium leakage conductances control the neuromodulatory state of the network and eliminate rhythmic bursting in the presence of spontaneous input (i.e., wake up the network). The aroused dLGN/PGN network model is subsequently stimulated by spatially homogeneous spontaneous retinal input or spatio-temporally patterned input consistent with the activity of X-type retinal ganglion cells during full-field or drifting grating visual stimulation. The throughput properties of this visually-driven dLGN/PGN network model are characterized and quantified as a function of stimulus parameters such as contrast, temporal frequency, and spatial frequency. During low-frequency oscillatory full-field stimulation, feedback inhibition from RE neurons often leads to TC neuron burst responses, while at high frequency tonic responses dominate. Depending on the average rate of stimulation, contrast level, and temporal frequency of modulation, the TC and RE cell bursts may or may not be phase-locked to the visual stimulus. During drifting-grating stimulation, phase-locked bursts often occur for sufficiently high contrast so long as the spatial period of the grating is not small compared to the synaptic footprint length, i.e., the spatial scale of the network connectivity.