Modeling and controlling the body in maladaptive ways: an active inference perspective on non-suicidal self-injury behaviors.

A significant number of persons engage in paradoxical behaviors, such as extreme food restriction (up to starvation) and non-suicidal self-injuries, especially during periods of rapid changes, such as adolescence. Here, we contextualize these and related paradoxical behavior within an active inference view of brain functions, which assumes that the brain forms predictive models of bodily variables, emotional experiences, and the embodied self and continuously strives to reduce the uncertainty of such models. We propose that not only in conditions of excessive or prolonged uncertainty, such as in clinical conditions, but also during pivotal periods of developmental transition, paradoxical behaviors might emerge as maladaptive strategies to reduce uncertainty-by "acting on the body"- soliciting salient perceptual and interoceptive sensations, such as pain or excessive levels of hunger. Although such strategies are maladaptive and run against our basic homeostatic imperatives, they might be functional not only to provide some short-term reward (e.g. relief from emotional distress)-as previously proposed-but also to reduce uncertainty and possibly to restore a coherent model of one's bodily experience and the self, affording greater confidence in who we are and what course of actions we should pursue.

The hippocampal formation as a hierarchical generative model supporting generative replay and continual learning.

We advance a novel computational theory of the hippocampal formation as a hierarchical generative model that organizes sequential experiences, such as rodent trajectories during spatial navigation, into coherent spatiotemporal contexts. We propose that the hippocampal generative model is endowed with inductive biases to identify individual items of experience (first hierarchical layer), organize them into sequences (second layer) and cluster them into maps (third layer). This theory entails a novel characterization of hippocampal reactivations as generative replay: the offline resampling of fictive sequences from the generative model, which supports the continual learning of multiple sequential experiences. We show that the model learns and efficiently retains multiple spatial navigation trajectories, by organizing them into spatial maps. Furthermore, the model reproduces flexible and prospective aspects of hippocampal dynamics that are challenging to explain within existing frameworks. This theory reconciles multiple roles of the hippocampal formation in map-based navigation, episodic memory and imagination.

Simulating homeostatic, allostatic and goal-directed forms of interoceptive control using active inference.

The adaptive regulation of bodily and interoceptive parameters, such as body temperature, thirst and hunger is a central problem for any biological organism. Here, we present a series of simulations using the framework of active inference to formally characterize interoceptive control and some of its dysfunctions. We start from the premise that the goal of interoceptive control is to minimize a discrepancy between expected and actual interoceptive sensations (i.e., a prediction error or free energy). Importantly, living organisms can achieve this goal by using various forms of interoceptive control: homeostatic, allostatic and goal-directed. We provide a computationally-guided analysis of these different forms of interoceptive control, by showing that they correspond to distinct generative models within Active inference. We discuss how these generative models can support empirical research through enabling fine-grained predictions about physiological and brain signals that may accompany both adaptive and maladaptive interoceptive control.

Perception and misperception of bodily symptoms from an active inference perspective: Modelling the case of panic disorder.

We advance a novel computational model that characterizes formally the ways we perceive or misperceive bodily symptoms, in the context of panic attacks. The computational model is grounded within the formal framework of Active Inference, which considers top-down prediction and attention dynamics as key to perceptual inference and action selection. In a series of simulations, we use the computational model to reproduce key facets of adaptive and maladaptive symptom perception: the ways we infer our bodily state by integrating prior information and somatic afferents; the ways we decide whether or not to attend to somatic channels; the ways we use the symptom inference to make decisions about taking or not taking a medicine; and the ways all the above processes can go awry, determining symptom misperception and ensuing maladaptive behaviors, such as hypervigilance or excessive medicine use. While recent existing theoretical treatments of psychopathological conditions focus on prediction-based perception (predictive coding), our computational model goes beyond them, in at least two ways. First, it includes action and attention selection dynamics that are disregarded in previous conceptualizations but are crucial to fully understand the phenomenology of bodily symptom perception and misperception. Second, it is a fully implemented model that generates specific (and personalized) quantitative predictions, thus going beyond previous qualitative frameworks. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

A framework to identify structured behavioral patterns within rodent spatial trajectories.

Animal behavior is highly structured. Yet, structured behavioral patterns-or "statistical ethograms"-are not immediately apparent from the full spatiotemporal data that behavioral scientists usually collect. Here, we introduce a framework to quantitatively characterize rodent behavior during spatial (e.g., maze) navigation, in terms of movement building blocks or motor primitives. The hypothesis that we pursue is that rodent behavior is characterized by a small number of motor primitives, which are combined over time to produce open-ended movements. We assume motor primitives to be organized in terms of two sparsity principles: each movement is controlled using a limited subset of motor primitives (sparse superposition) and each primitive is active only for time-limited, time-contiguous portions of movements (sparse activity). We formalize this hypothesis using a sparse dictionary learning method, which we use to extract motor primitives from rodent position and velocity data collected during spatial navigation, and successively to reconstruct past trajectories and predict novel ones. Three main results validate our approach. First, rodent behavioral trajectories are robustly reconstructed from incomplete data, performing better than approaches based on standard dimensionality reduction methods, such as principal component analysis, or single sparsity. Second, the motor primitives extracted during one experimental session generalize and afford the accurate reconstruction of rodent behavior across successive experimental sessions in the same or in modified mazes. Third, in our approach the number of motor primitives associated with each maze correlates with independent measures of maze complexity, hence showing that our formalism is sensitive to essential aspects of task structure. The framework introduced here can be used by behavioral scientists and neuroscientists as an aid for behavioral and neural data analysis. Indeed, the extracted motor primitives enable the quantitative characterization of the complexity and similarity between different mazes and behavioral patterns across multiple trials (i.e., habit formation). We provide example uses of this computational framework, showing how it can be used to identify behavioural effects of maze complexity, analyze stereotyped behavior, classify behavioral choices and predict place and grid cell displacement in novel environments.

Social epistemic actions.

We consider the ways humans engage in social epistemic actions, to guide each other's attention, prediction, and learning processes towards salient information, at the timescale of online social interaction and joint action. This parallels the active guidance of other's attention, prediction, and learning processes at the longer timescale of niche construction and cultural practices, as discussed in the target article.

Tracking the Time Course of Bayesian Inference With Event-Related Potentials:A Study Using the Central Cue Posner Paradigm.

In this study, we asked whether the event-related potentials associated to cue and target stimuli of a Central Cue Posner Paradigm (CCPP) may encode key parameters of Bayesian inference - prior expectation and surprise - on a trial-by-trial basis. Thirty-two EEG channel were recorded in a sample of 19 young adult subjects while performing a CCPP, in which a cue indicated (validly or invalidly) the position of an incoming auditory target. Three different types of blocks with validities of 50%, 64%, and 88%, respectively, were presented. Estimates of prior expectation and surprise were obtained on a trial-by-trial basis from participants' responses, using a computational model implementing Bayesian learning. These two values were correlated on a trial-by-trial basis with the EEG values in all the electrodes and time bins. Therefore, a Spearman correlation metrics of the relationship between Bayesian parameters and the EEG was obtained. We report that the surprise parameter was able to classify the different validity blocks. Furthermore, the prior expectation parameter showed a significant correlation with the EEG in the cue-target period, in which the Contingent Negative Variation develops. Finally, in the post-target period the surprise parameter showed a significant correlation in the latencies and electrodes in which different event-related potentials are induced. Our results suggest that Bayesian parameters are coded in the EEG signals; and namely, the CNV would be related to prior expectation, while the post-target components P2a, P2, P3a, P3b, and SW would be related to surprise. This study thus provides novel support to the idea that human electrophysiological neural activity may implement a (Bayesian) predictive processing scheme.

Problem Solving as Probabilistic Inference with Subgoaling: Explaining Human Successes and Pitfalls in the Tower of Hanoi.

How do humans and other animals face novel problems for which predefined solutions are not available? Human problem solving links to flexible reasoning and inference rather than to slow trial-and-error learning. It has received considerable attention since the early days of cognitive science, giving rise to well known cognitive architectures such as SOAR and ACT-R, but its computational and brain mechanisms remain incompletely known. Furthermore, it is still unclear whether problem solving is a "specialized" domain or module of cognition, in the sense that it requires computations that are fundamentally different from those supporting perception and action systems. Here we advance a novel view of human problem solving as probabilistic inference with subgoaling. In this perspective, key insights from cognitive architectures are retained such as the importance of using subgoals to split problems into subproblems. However, here the underlying computations use probabilistic inference methods analogous to those that are increasingly popular in the study of perception and action systems. To test our model we focus on the widely used Tower of Hanoi (ToH) task, and show that our proposed method can reproduce characteristic idiosyncrasies of human problem solvers: their sensitivity to the "community structure" of the ToH and their difficulties in executing so-called "counterintuitive" movements. Our analysis reveals that subgoals have two key roles in probabilistic inference and problem solving. First, prior beliefs on (likely) useful subgoals carve the problem space and define an implicit metric for the problem at hand-a metric to which humans are sensitive. Second, subgoals are used as waypoints in the probabilistic problem solving inference and permit to find effective solutions that, when unavailable, lead to problem solving deficits. Our study thus suggests that a probabilistic inference scheme enhanced with subgoals provides a comprehensive framework to study problem solving and its deficits.

Divide et impera: subgoaling reduces the complexity of probabilistic inference and problem solving.

It has long been recognized that humans (and possibly other animals) usually break problems down into smaller and more manageable problems using subgoals. Despite a general consensus that subgoaling helps problem solving, it is still unclear what the mechanisms guiding online subgoal selection are during the solution of novel problems for which predefined solutions are not available. Under which conditions does subgoaling lead to optimal behaviour? When is subgoaling better than solving a problem from start to finish? Which is the best number and sequence of subgoals to solve a given problem? How are these subgoals selected during online inference? Here, we present a computational account of subgoaling in problem solving. Following Occam's razor, we propose that good subgoals are those that permit planning solutions and controlling behaviour using less information resources, thus yielding parsimony in inference and control. We implement this principle using approximate probabilistic inference: subgoals are selected using a sampling method that considers the descriptive complexity of the resulting sub-problems. We validate the proposed method using a standard reinforcement learning benchmark (four-rooms scenario) and show that the proposed method requires less inferential steps and permits selecting more compact control programs compared to an equivalent procedure without subgoaling. Furthermore, we show that the proposed method offers a mechanistic explanation of the neuronal dynamics found in the prefrontal cortex of monkeys that solve planning problems. Our computational framework provides a novel integrative perspective on subgoaling and its adaptive advantages for planning, control and learning, such as for example lowering cognitive effort and working memory load.