Your browser doesn't support javascript.
loading
Show: 20 | 50 | 100
Results 1 - 20 de 93
Filter
1.
PLoS Comput Biol ; 20(4): e1011183, 2024 Apr.
Article in English | MEDLINE | ID: mdl-38557984

ABSTRACT

One of the key problems the brain faces is inferring the state of the world from a sequence of dynamically changing stimuli, and it is not yet clear how the sensory system achieves this task. A well-established computational framework for describing perceptual processes in the brain is provided by the theory of predictive coding. Although the original proposals of predictive coding have discussed temporal prediction, later work developing this theory mostly focused on static stimuli, and key questions on neural implementation and computational properties of temporal predictive coding networks remain open. Here, we address these questions and present a formulation of the temporal predictive coding model that can be naturally implemented in recurrent networks, in which activity dynamics rely only on local inputs to the neurons, and learning only utilises local Hebbian plasticity. Additionally, we show that temporal predictive coding networks can approximate the performance of the Kalman filter in predicting behaviour of linear systems, and behave as a variant of a Kalman filter which does not track its own subjective posterior variance. Importantly, temporal predictive coding networks can achieve similar accuracy as the Kalman filter without performing complex mathematical operations, but just employing simple computations that can be implemented by biological networks. Moreover, when trained with natural dynamic inputs, we found that temporal predictive coding can produce Gabor-like, motion-sensitive receptive fields resembling those observed in real neurons in visual areas. In addition, we demonstrate how the model can be effectively generalized to nonlinear systems. Overall, models presented in this paper show how biologically plausible circuits can predict future stimuli and may guide research on understanding specific neural circuits in brain areas involved in temporal prediction.


Subject(s)
Brain , Models, Neurological , Brain/physiology , Learning , Neurons/physiology
2.
PLoS Comput Biol ; 20(4): e1011516, 2024 Apr.
Article in English | MEDLINE | ID: mdl-38626219

ABSTRACT

When facing an unfamiliar environment, animals need to explore to gain new knowledge about which actions provide reward, but also put the newly acquired knowledge to use as quickly as possible. Optimal reinforcement learning strategies should therefore assess the uncertainties of these action-reward associations and utilise them to inform decision making. We propose a novel model whereby direct and indirect striatal pathways act together to estimate both the mean and variance of reward distributions, and mesolimbic dopaminergic neurons provide transient novelty signals, facilitating effective uncertainty-driven exploration. We utilised electrophysiological recording data to verify our model of the basal ganglia, and we fitted exploration strategies derived from the neural model to data from behavioural experiments. We also compared the performance of directed exploration strategies inspired by our basal ganglia model with other exploration algorithms including classic variants of upper confidence bound (UCB) strategy in simulation. The exploration strategies inspired by the basal ganglia model can achieve overall superior performance in simulation, and we found qualitatively similar results in fitting model to behavioural data compared with the fitting of more idealised normative models with less implementation level detail. Overall, our results suggest that transient dopamine levels in the basal ganglia that encode novelty could contribute to an uncertainty representation which efficiently drives exploration in reinforcement learning.


Subject(s)
Basal Ganglia , Dopamine , Models, Neurological , Reward , Dopamine/metabolism , Dopamine/physiology , Uncertainty , Animals , Basal Ganglia/physiology , Exploratory Behavior/physiology , Reinforcement, Psychology , Dopaminergic Neurons/physiology , Computational Biology , Computer Simulation , Male , Algorithms , Decision Making/physiology , Behavior, Animal/physiology , Rats
3.
Nat Neurosci ; 27(2): 348-358, 2024 Feb.
Article in English | MEDLINE | ID: mdl-38172438

ABSTRACT

For both humans and machines, the essence of learning is to pinpoint which components in its information processing pipeline are responsible for an error in its output, a challenge that is known as 'credit assignment'. It has long been assumed that credit assignment is best solved by backpropagation, which is also the foundation of modern machine learning. Here, we set out a fundamentally different principle on credit assignment called 'prospective configuration'. In prospective configuration, the network first infers the pattern of neural activity that should result from learning, and then the synaptic weights are modified to consolidate the change in neural activity. We demonstrate that this distinct mechanism, in contrast to backpropagation, (1) underlies learning in a well-established family of models of cortical circuits, (2) enables learning that is more efficient and effective in many contexts faced by biological organisms and (3) reproduces surprising patterns of neural activity and behavior observed in diverse human and rat learning experiments.


Subject(s)
Machine Learning , Neural Networks, Computer , Humans , Rats , Animals , Prospective Studies , Neuronal Plasticity
4.
Brain Stimul ; 16(5): 1412-1424, 2023.
Article in English | MEDLINE | ID: mdl-37683763

ABSTRACT

OBJECTIVES: The exact mechanisms of deep brain stimulation (DBS) are still an active area of investigation, in spite of its clinical successes. This is due in part to the lack of understanding of the effects of stimulation on neuronal rhythms. Entrainment of brain oscillations has been hypothesised as a potential mechanism of neuromodulation. A better understanding of entrainment might further inform existing methods of continuous DBS, and help refine algorithms for adaptive methods. The purpose of this study is to develop and test a theoretical framework to predict entrainment of cortical rhythms to DBS across a wide range of stimulation parameters. MATERIALS AND METHODS: We fit a model of interacting neural populations to selected features characterising PD patients' off-stimulation finely-tuned gamma rhythm recorded through electrocorticography. Using the fitted models, we predict basal ganglia DBS parameters that would result in 1:2 entrainment, a special case of sub-harmonic entrainment observed in patients and predicted by theory. RESULTS: We show that the neural circuit models fitted to patient data exhibit 1:2 entrainment when stimulation is provided across a range of stimulation parameters. Furthermore, we verify key features of the region of 1:2 entrainment in the stimulation frequency/amplitude space with follow-up recordings from the same patients, such as the loss of 1:2 entrainment above certain stimulation amplitudes. CONCLUSION: Our results reveal that continuous, constant frequency DBS in patients may lead to nonlinear patterns of neuronal entrainment across stimulation parameters, and that these responses can be predicted by modelling. Should entrainment prove to be an important mechanism of therapeutic stimulation, our modelling framework may reduce the parameter space that clinicians must consider when programming devices for optimal benefit.


Subject(s)
Deep Brain Stimulation , Parkinson Disease , Humans , Parkinson Disease/therapy , Deep Brain Stimulation/methods , Basal Ganglia , Physical Therapy Modalities , Electrocorticography
5.
PLoS Biol ; 21(6): e3002140, 2023 06.
Article in English | MEDLINE | ID: mdl-37262014

ABSTRACT

Adapting actions to changing goals and environments is central to intelligent behavior. There is evidence that the basal ganglia play a crucial role in reinforcing or adapting actions depending on their outcome. However, the corresponding electrophysiological correlates in the basal ganglia and the extent to which these causally contribute to action adaptation in humans is unclear. Here, we recorded electrophysiological activity and applied bursts of electrical stimulation to the subthalamic nucleus, a core area of the basal ganglia, in 16 patients with Parkinson's disease (PD) on medication using temporarily externalized deep brain stimulation (DBS) electrodes. Patients as well as 16 age- and gender-matched healthy participants attempted to produce forces as close as possible to a target force to collect a maximum number of points. The target force changed over trials without being explicitly shown on the screen so that participants had to infer target force based on the feedback they received after each movement. Patients and healthy participants were able to adapt their force according to the feedback they received (P < 0.001). At the neural level, decreases in subthalamic beta (13 to 30 Hz) activity reflected poorer outcomes and stronger action adaptation in 2 distinct time windows (Pcluster-corrected < 0.05). Stimulation of the subthalamic nucleus reduced beta activity and led to stronger action adaptation if applied within the time windows when subthalamic activity reflected action outcomes and adaptation (Pcluster-corrected < 0.05). The more the stimulation volume was connected to motor cortex, the stronger was this behavioral effect (Pcorrected = 0.037). These results suggest that dynamic modulation of the subthalamic nucleus and interconnected cortical areas facilitates adaptive behavior.


Subject(s)
Deep Brain Stimulation , Parkinson Disease , Subthalamic Nucleus , Humans , Subthalamic Nucleus/physiology , Deep Brain Stimulation/methods , Parkinson Disease/therapy , Basal Ganglia , Adaptation, Psychological
6.
Brain Behav ; 13(5): e2978, 2023 05.
Article in English | MEDLINE | ID: mdl-37016956

ABSTRACT

INTRODUCTION: We assess risks differently when they are explicitly described, compared to when we learn directly from experience, suggesting dissociable decision-making systems. Our needs, such as hunger, could globally affect our risk preferences, but do they affect described and learned risks equally? On one hand, decision-making from descriptions is often considered flexible and context sensitive, and might therefore be modulated by metabolic needs. On the other hand, preferences learned through reinforcement might be more strongly coupled to biological drives. METHOD: Thirty-two healthy participants (females: 20, mean age: 25.6 ± 6.5 years) with a normal weight (Body Mass Index: 22.9 ± 3.2 kg/m2 ) were tested in a within-subjects counterbalanced, randomized crossover design for the effects of hunger on two separate risk-taking tasks. We asked participants to choose between two options with different risks to obtain monetary outcomes. In one task, the outcome probabilities were described numerically, whereas in a second task, they were learned. RESULT: In agreement with previous studies, we found that rewarding contexts induced risk-aversion when risks were explicitly described (F1,31  = 55.01, p < .0001, ηp 2  = .64), but risk-seeking when they were learned through experience (F1,31  = 10.28, p < .003, ηp 2  = .25). Crucially, hunger attenuated these contextual biases, but only for learned risks (F1,31  = 8.38, p < .007, ηp 2  = .21). CONCLUSION: The results suggest that our metabolic state determines risk-taking biases when we lack explicit descriptions.


Subject(s)
Gambling , Adult , Female , Humans , Young Adult , Decision Making , Hunger , Probability , Risk-Taking , Stomach , Cross-Over Studies
7.
PLoS Comput Biol ; 19(4): e1010719, 2023 04.
Article in English | MEDLINE | ID: mdl-37058541

ABSTRACT

The computational principles adopted by the hippocampus in associative memory (AM) tasks have been one of the most studied topics in computational and theoretical neuroscience. Recent theories suggested that AM and the predictive activities of the hippocampus could be described within a unitary account, and that predictive coding underlies the computations supporting AM in the hippocampus. Following this theory, a computational model based on classical hierarchical predictive networks was proposed and was shown to perform well in various AM tasks. However, this fully hierarchical model did not incorporate recurrent connections, an architectural component of the CA3 region of the hippocampus that is crucial for AM. This makes the structure of the model inconsistent with the known connectivity of CA3 and classical recurrent models such as Hopfield Networks, which learn the covariance of inputs through their recurrent connections to perform AM. Earlier PC models that learn the covariance information of inputs explicitly via recurrent connections seem to be a solution to these issues. Here, we show that although these models can perform AM, they do it in an implausible and numerically unstable way. Instead, we propose alternatives to these earlier covariance-learning predictive coding networks, which learn the covariance information implicitly and plausibly, and can use dendritic structures to encode prediction errors. We show analytically that our proposed models are perfectly equivalent to the earlier predictive coding model learning covariance explicitly, and encounter no numerical issues when performing AM tasks in practice. We further show that our models can be combined with hierarchical predictive coding networks to model the hippocampo-neocortical interactions. Our models provide a biologically plausible approach to modelling the hippocampal network, pointing to a potential computational mechanism during hippocampal memory formation and recall, which employs both predictive coding and covariance learning based on the recurrent network structure of the hippocampus.


Subject(s)
Hippocampus , Learning , Mental Recall , Conditioning, Classical , Models, Neurological
8.
J Neural Eng ; 20(2)2023 03 07.
Article in English | MEDLINE | ID: mdl-36880684

ABSTRACT

Objective.While brain stimulation therapies such as deep brain stimulation for Parkinson's disease (PD) can be effective, they have yet to reach their full potential across neurological disorders. Entraining neuronal rhythms using rhythmic brain stimulation has been suggested as a new therapeutic mechanism to restore neurotypical behaviour in conditions such as chronic pain, depression, and Alzheimer's disease. However, theoretical and experimental evidence indicate that brain stimulation can also entrain neuronal rhythms at sub- and super-harmonics, far from the stimulation frequency. Crucially, these counterintuitive effects could be harmful to patients, for example by triggering debilitating involuntary movements in PD. We therefore seek a principled approach to selectively promote rhythms close to the stimulation frequency, while avoiding potential harmful effects by preventing entrainment at sub- and super-harmonics.Approach.Our open-loop approach to selective entrainment, dithered stimulation, consists in adding white noise to the stimulation period.Main results.We theoretically establish the ability of dithered stimulation to selectively entrain a given brain rhythm, and verify its efficacy in simulations of uncoupled neural oscillators, and networks of coupled neural oscillators. Furthermore, we show that dithered stimulation can be implemented in neurostimulators with limited capabilities by toggling within a finite set of stimulation frequencies.Significance.Likely implementable across a variety of existing brain stimulation devices, dithering-based selective entrainment has potential to enable new brain stimulation therapies, as well as new neuroscientific research exploiting its ability to modulate higher-order entrainment.


Subject(s)
Alzheimer Disease , Parkinson Disease , Humans , Stereotaxic Techniques , Brain , Parkinson Disease/therapy
9.
J Math Psychol ; 117: 102815, 2023 Dec.
Article in English | MEDLINE | ID: mdl-38188903

ABSTRACT

We introduce a new approach to modelling decision confidence, with the aim of enabling computationally cheap predictions while taking into account, and thereby exploiting, trial-by-trial variability in stochastically fluctuating stimuli. Using the framework of the drift diffusion model of decision making, along with time-dependent thresholds and the idea of a Bayesian confidence readout, we derive expressions for the probability distribution over confidence reports. In line with current models of confidence, the derivations allow for the accumulation of "pipeline" evidence that has been received but not processed by the time of response, the effect of drift rate variability, and metacognitive noise. The expressions are valid for stimuli that change over the course of a trial with normally-distributed fluctuations in the evidence they provide. A number of approximations are made to arrive at the final expressions, and we test all approximations via simulation. The derived expressions contain only a small number of standard functions, and require evaluating only once per trial, making trial-by-trial modelling of confidence data in stochastically fluctuating stimuli tasks more feasible. We conclude by using the expressions to gain insight into the confidence of optimal observers, and empirically observed patterns.

10.
Adv Neural Inf Process Syst ; 36: 44341-44355, 2023.
Article in English | MEDLINE | ID: mdl-38606302

ABSTRACT

Forming accurate memory of sequential stimuli is a fundamental function of biological agents. However, the computational mechanism underlying sequential memory in the brain remains unclear. Inspired by neuroscience theories and recent successes in applying predictive coding (PC) to static memory tasks, in this work we propose a novel PC-based model for sequential memory, called temporal predictive coding (tPC). We show that our tPC models can memorize and retrieve sequential inputs accurately with a biologically plausible neural implementation. Importantly, our analytical study reveals that tPC can be viewed as a classical Asymmetric Hopfield Network (AHN) with an implicit statistical whitening process, which leads to more stable performance in sequential memory tasks of structured inputs. Moreover, we find that tPC exhibits properties consistent with behavioral observations and theories in neuroscience, thereby strengthening its biological relevance. Our work establishes a possible computational mechanism underlying sequential memory in the brain that can also be theoretically interpreted using existing memory model frameworks.

11.
Nat Commun ; 13(1): 7530, 2022 12 07.
Article in English | MEDLINE | ID: mdl-36476581

ABSTRACT

To optimally adjust our behavior to changing environments we need to both adjust the speed of our decisions and movements. Yet little is known about the extent to which these processes are controlled by common or separate mechanisms. Furthermore, while previous evidence from computational models and empirical studies suggests that the basal ganglia play an important role during adjustments of decision-making, it remains unclear how this is implemented. Leveraging the opportunity to directly access the subthalamic nucleus of the basal ganglia in humans undergoing deep brain stimulation surgery, we here combine invasive electrophysiological recordings, electrical stimulation and computational modelling of perceptual decision-making. We demonstrate that, while similarities between subthalamic control of decision- and movement speed exist, the causal contribution of the subthalamic nucleus to these processes can be disentangled. Our results show that the basal ganglia independently control the speed of decisions and movement for each hemisphere during adaptive behavior.


Subject(s)
Basal Ganglia , Humans
12.
J Neurooncol ; 160(3): 753-761, 2022 Dec.
Article in English | MEDLINE | ID: mdl-36449256

ABSTRACT

PURPOSE: Despite the improvement in treatment and prognosis of primary central nervous system lymphoma (PCNSL) over the last decades, the 5-year survival rate is approximately 30%; thus, new therapeutic approaches are needed to improve patient survival. The study's aim was to evaluate the role of surgical resection of PCNSL. METHODS: Primary outcomes were the overall survival (OS) and progression-free survival (PFS) of patients with PCNSL who underwent surgical resection versus biopsy alone. The meta-analysis was conducted to calculate pooled hazard ratios (HRs) under a random-effects model for the time-to-event variables. The odds ratios (ORs) were calculated for binary, secondary outcome parameters. RESULTS: Seven studies (n = 1046) were included. We found that surgical resection was associated with significantly better OS (HR 0.63 [95% CI 0.51-0.77]) when compared with biopsy. PFS was also significantly improved (HR 0.64 [95% CI 0.49-0.85]) in patients who underwent resection compared with those who underwent biopsy. The heterogeneity for OS and PFS was low (I2 = 7% and 24%, respectively). We also found that patients who underwent biopsy more often had multiple (OR 0.38 [95% CI 0.19-0.79]) or deep-seated (OR 0.20 [95% CI 0.12-0.34]) lesions compared with those who underwent surgical resection. There were no significant differences in chemotherapy or radiotherapy use or the occurrence of postoperative complications between the two groups. CONCLUSION: In selected patients, surgical resection of PCNSL is associated with significantly better overall survival and progression-free survival compared with biopsy alone.


Subject(s)
Central Nervous System Neoplasms , Lymphoma , Humans , Progression-Free Survival , Biopsy , Combined Modality Therapy , Lymphoma/surgery , Lymphoma/drug therapy , Central Nervous System
13.
PLoS Comput Biol ; 18(5): e1009816, 2022 05.
Article in English | MEDLINE | ID: mdl-35622863

ABSTRACT

To accurately predict rewards associated with states or actions, the variability of observations has to be taken into account. In particular, when the observations are noisy, the individual rewards should have less influence on tracking of average reward, and the estimate of the mean reward should be updated to a smaller extent after each observation. However, it is not known how the magnitude of the observation noise might be tracked and used to control prediction updates in the brain reward system. Here, we introduce a new model that uses simple, tractable learning rules that track the mean and standard deviation of reward, and leverages prediction errors scaled by uncertainty as the central feedback signal. We show that the new model has an advantage over conventional reinforcement learning models in a value tracking task, and approaches a theoretic limit of performance provided by the Kalman filter. Further, we propose a possible biological implementation of the model in the basal ganglia circuit. In the proposed network, dopaminergic neurons encode reward prediction errors scaled by standard deviation of rewards. We show that such scaling may arise if the striatal neurons learn the standard deviation of rewards and modulate the activity of dopaminergic neurons. The model is consistent with experimental findings concerning dopamine prediction error scaling relative to reward magnitude, and with many features of striatal plasticity. Our results span across the levels of implementation, algorithm, and computation, and might have important implications for understanding the dopaminergic prediction error signal and its relation to adaptive and effective learning.


Subject(s)
Basal Ganglia , Learning , Basal Ganglia/physiology , Dopamine/physiology , Learning/physiology , Reinforcement, Psychology , Reward , Uncertainty
14.
J Neurosci ; 42(23): 4681-4692, 2022 06 08.
Article in English | MEDLINE | ID: mdl-35501153

ABSTRACT

Making accurate decisions often involves the integration of current and past evidence. Here, we examine the neural correlates of conflict and evidence integration during sequential decision-making. Female and male human patients implanted with deep-brain stimulation (DBS) electrodes and age-matched and gender-matched healthy controls performed an expanded judgment task, in which they were free to choose how many cues to sample. Behaviorally, we found that while patients sampled numerically more cues, they were less able to integrate evidence and showed suboptimal performance. Using recordings of magnetoencephalography (MEG) and local field potentials (LFPs; in patients) in the subthalamic nucleus (STN), we found that ß oscillations signaled conflict between cues within a sequence. Following cues that differed from previous cues, ß power in the STN and cortex first decreased and then increased. Importantly, the conflict signal in the STN outlasted the cortical one, carrying over to the next cue in the sequence. Furthermore, after a conflict, there was an increase in coherence between the dorsal premotor cortex and STN in the ß band. These results extend our understanding of cortico-subcortical dynamics of conflict processing, and do so in a context where evidence must be accumulated in discrete steps, much like in real life. Thus, the present work leads to a more nuanced picture of conflict monitoring systems in the brain and potential changes because of disease.SIGNIFICANCE STATEMENT Decision-making often involves the integration of multiple pieces of information over time to make accurate predictions. We simultaneously recorded whole-head magnetoencephalography (MEG) and local field potentials (LFPs) from the human subthalamic nucleus (STN) in a novel task which required integrating sequentially presented pieces of evidence. Our key finding is prolonged ß oscillations in the STN, with a concurrent increase in communication with frontal cortex, when presented with conflicting information. These neural effects reflect the behavioral profile of reduced tendency to respond after conflict, as well as relate to suboptimal cue integration in patients, which may be directly linked to clinically reported side-effects of deep-brain stimulation (DBS) such as impaired decision-making and impulsivity.


Subject(s)
Deep Brain Stimulation , Motor Cortex , Parkinson Disease , Subthalamic Nucleus , Beta Rhythm , Deep Brain Stimulation/methods , Female , Humans , Magnetoencephalography , Male , Motor Cortex/physiology , Parkinson Disease/therapy , Subthalamic Nucleus/physiology
15.
Adv Neural Inf Process Syst ; 35: 38232-38244, 2022 Nov.
Article in English | MEDLINE | ID: mdl-37090087

ABSTRACT

Training with backpropagation (BP) in standard deep learning consists of two main steps: a forward pass that maps a data point to its prediction, and a backward pass that propagates the error of this prediction back through the network. This process is highly effective when the goal is to minimize a specific objective function. However, it does not allow training on networks with cyclic or backward connections. This is an obstacle to reaching brain-like capabilities, as the highly complex heterarchical structure of the neural connections in the neocortex are potentially fundamental for its effectiveness. In this paper, we show how predictive coding (PC), a theory of information processing in the cortex, can be used to perform inference and learning on arbitrary graph topologies. We experimentally show how this formulation, called PC graphs, can be used to flexibly perform different tasks with the same network by simply stimulating specific neurons. This enables the model to be queried on stimuli with different structures, such as partial images, images with labels, or images without labels. We conclude by investigating how the topology of the graph influences the final performance, and comparing against simple baselines trained with BP.

16.
Proc AAAI Conf Artif Intell ; 36(7): 8150-8158, 2022 Jun 28.
Article in English | MEDLINE | ID: mdl-37205168

ABSTRACT

Deep learning has redefined AI thanks to the rise of artificial neural networks, which are inspired by neuronal networks in the brain. Through the years, these interactions between AI and neuroscience have brought immense benefits to both fields, allowing neural networks to be used in a plethora of applications. Neural networks use an efficient implementation of reverse differentiation, called backpropagation (BP). This algorithm, however, is often criticized for its biological implausibility (e.g., lack of local update rules for the parameters). Therefore, biologically plausible learning methods that rely on predictive coding (PC), a framework for describing information processing in the brain, are increasingly studied. Recent works prove that these methods can approximate BP up to a certain margin on multilayer perceptrons (MLPs), and asymptotically on any other complex model, and that zerodivergence inference learning (Z-IL), a variant of PC, is able to exactly implement BP on MLPs. However, the recent literature shows also that there is no biologically plausible method yet that can exactly replicate the weight update of BP on complex models. To fill this gap, in this paper, we generalize (PC and) Z-IL by directly defining it on computational graphs, and show that it can perform exact reverse differentiation. What results is the first PC (and so biologically plausible) algorithm that is equivalent to BP in the way of updating parameters on any neural network, providing a bridge between the interdisciplinary research of neuroscience and deep learning. Furthermore, the above results in particular also immediately provide a novel local and parallel implementation of BP.

17.
Neural Comput ; 34(2): 307-337, 2022 01 14.
Article in English | MEDLINE | ID: mdl-34758486

ABSTRACT

Reinforcement learning involves updating estimates of the value of states and actions on the basis of experience. Previous work has shown that in humans, reinforcement learning exhibits a confirmatory bias: when the value of a chosen option is being updated, estimates are revised more radically following positive than negative reward prediction errors, but the converse is observed when updating the unchosen option value estimate. Here, we simulate performance on a multi-arm bandit task to examine the consequences of a confirmatory bias for reward harvesting. We report a paradoxical finding: that confirmatory biases allow the agent to maximize reward relative to an unbiased updating rule. This principle holds over a wide range of experimental settings and is most influential when decisions are corrupted by noise. We show that this occurs because on average, confirmatory biases lead to overestimating the value of more valuable bandits and underestimating the value of less valuable bandits, rendering decisions overall more robust in the face of noise. Our results show how apparently suboptimal learning rules can in fact be reward maximizing if decisions are made with finite computational precision.


Subject(s)
Learning , Reinforcement, Psychology , Bias , Decision Making , Humans , Reward
18.
Evid Based Ment Health ; 25(2): 77-83, 2022 05.
Article in English | MEDLINE | ID: mdl-34810175

ABSTRACT

INTRODUCTION: Clinical depression is usually treated in primary care with psychological therapies and antidepressant medication. However, when patients do not respond to at least two or more antidepressants within a depressive episode, they are considered to have treatment resistant depression (TRD). Previous small randomised controlled trials suggested that pramipexole, a dopamine D2/3 receptor agonist, may be effective for treating patients with unipolar and bipolar depression as it is known to influence motivational drive and reward processing. PAX-D will compare the effects of pramipexole vs placebo when added to current antidepressant medication for people with TRD. Additionally, PAX-D will investigate the mechanistic effect of pramipexole on reward sensitivity using a probabilistic decision-making task. METHODS AND ANALYSIS: PAX-D will assess effectiveness in the short- term (during the first 12 weeks) and in the longer-term (48 weeks) in patients with TRD from the UK. The primary outcome will be change in self-reported depressive symptoms from baseline to week 12 post-randomisation measured using the Quick Inventory of Depressive Symptomatology Self-Report (QIDS-SR16). Performance on the decision-making task will be measured at week 0, week 2 and week 12. Secondary outcomes include anhedonia, anxiety and health economic measures including quality of life, capability, well-being and costs. PAX-D will also assess the adverse effects of pramipexole including impulse control difficulties. DISCUSSION: Pramipexole is a promising augmentation agent for TRD and may be a useful addition to existing treatment regimes. PAX-D will assess its effectiveness and test for a potential mechanism of action in patients with TRD. TRIAL REGISTRATION NUMBER: ISRCTN84666271.


Subject(s)
Depressive Disorder, Major , Depressive Disorder, Treatment-Resistant , Antidepressive Agents/adverse effects , Depression/drug therapy , Depressive Disorder, Major/drug therapy , Depressive Disorder, Treatment-Resistant/drug therapy , Humans , Pramipexole/pharmacology , Pramipexole/therapeutic use , Quality of Life , Randomized Controlled Trials as Topic
19.
Proc Mach Learn Res ; 162: 15561-15583, 2022 Jul.
Article in English | MEDLINE | ID: mdl-36751405

ABSTRACT

A large number of neural network models of associative memory have been proposed in the literature. These include the classical Hopfield networks (HNs), sparse distributed memories (SDMs), and more recently the modern continuous Hopfield networks (MCHNs), which possess close links with self-attention in machine learning. In this paper, we propose a general framework for understanding the operation of such memory networks as a sequence of three operations: similarity, separation, and projection. We derive all these memory models as instances of our general framework with differing similarity and separation functions. We extend the mathematical framework of Krotov & Hopfield (2020) to express general associative memory models using neural network dynamics with local computation, and derive a general energy function that is a Lyapunov function of the dynamics. Finally, using our framework, we empirically investigate the capacity of using different similarity functions for these associative memory models, beyond the dot product similarity measure, and demonstrate empirically that Euclidean or Manhattan distance similarity metrics perform substantially better in practice on many tasks, enabling a more robust retrieval and higher memory capacity than existing models.

20.
Front Neurosci ; 15: 734265, 2021.
Article in English | MEDLINE | ID: mdl-34630021

ABSTRACT

Circadian and other physiological rhythms play a key role in both normal homeostasis and disease processes. Such is the case of circadian and infradian seizure patterns observed in epilepsy. However, these rhythms are not fully exploited in the design of active implantable medical devices. In this paper we explore a new implantable stimulator that implements chronotherapy as a feedforward input to supplement both open-loop and closed-loop methods. This integrated algorithm allows for stimulation to be adjusted to the ultradian, circadian and infradian patterns observed in patients through slowly-varying temporal adjustments of stimulation and algorithm sub-components, while also enabling adaption of stimulation based on immediate physiological needs such as a breakthrough seizure or change of posture. Embedded physiological sensors in the stimulator can be used to refine the baseline stimulation circadian pattern as a "digital zeitgeber," i.e., a source of stimulus that entrains or synchronizes the subject's natural rhythms. This algorithmic approach is tested on a canine with severe drug-resistant idiopathic generalized epilepsy exhibiting a characteristic diurnal pattern correlated with sleep-wake cycles. Prior to implantation, the canine's cluster seizures evolved to status epilepticus (SE) and required emergency pharmacological intervention. The cranially-mounted system was fully-implanted bilaterally into the centromedian nucleus of the thalamus. Using combinations of time-based modulation, thalamocortical rhythm-specific tuning of frequency parameters as well as fast-adaptive modes based on activity, the canine experienced no further SE events post-implant as of the time of writing (7 months). Importantly, no significant cluster seizures have been observed either, allowing the reduction of rescue medication. The use of digitally-enabled chronotherapy as a feedforward signal to augment adaptive neurostimulators could prove a useful algorithmic method in conditions where sensitivity to temporal patterns are characteristics of the disease state, providing a novel mechanism for tailoring a more patient-specific therapy approach.

SELECTION OF CITATIONS
SEARCH DETAIL
...