Human Belief State-Based Exploration and Exploitation in an Information-Selective Symmetric Reversal Bandit Task.

Humans often face sequential decision-making problems, in which information about the environmental reward structure is detached from rewards for a subset of actions. In the current exploratory study, we introduce an information-selective symmetric reversal bandit task to model such situations and obtained choice data on this task from 24 participants. To arbitrate between different decision-making strategies that participants may use on this task, we developed a set of probabilistic agent-based behavioral models, including exploitative and explorative Bayesian agents, as well as heuristic control agents. Upon validating the model and parameter recovery properties of our model set and summarizing the participants' choice data in a descriptive way, we used a maximum likelihood approach to evaluate the participants' choice data from the perspective of our model set. In brief, we provide quantitative evidence that participants employ a belief state-based hybrid explorative-exploitative strategy on the information-selective symmetric reversal bandit task, lending further support to the finding that humans are guided by their subjective uncertainty when solving exploration-exploitation dilemmas. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s42113-021-00112-3.

Neural surprise in somatosensory Bayesian learning.

Tracking statistical regularities of the environment is important for shaping human behavior and perception. Evidence suggests that the brain learns environmental dependencies using Bayesian principles. However, much remains unknown about the employed algorithms, for somesthesis in particular. Here, we describe the cortical dynamics of the somatosensory learning system to investigate both the form of the generative model as well as its neural surprise signatures. Specifically, we recorded EEG data from 40 participants subjected to a somatosensory roving-stimulus paradigm and performed single-trial modeling across peri-stimulus time in both sensor and source space. Our Bayesian model selection procedure indicates that evoked potentials are best described by a non-hierarchical learning model that tracks transitions between observations using leaky integration. From around 70ms post-stimulus onset, secondary somatosensory cortices are found to represent confidence-corrected surprise as a measure of model inadequacy. Indications of Bayesian surprise encoding, reflecting model updating, are found in primary somatosensory cortex from around 140ms. This dissociation is compatible with the idea that early surprise signals may control subsequent model update rates. In sum, our findings support the hypothesis that early somatosensory processing reflects Bayesian perceptual learning and contribute to an understanding of its underlying mechanisms.

Brain-Behavior Associations for Risk Taking Depend on the Measures Used to Capture Individual Differences.

Maladaptive risk taking can have severe individual and societal consequences; thus, individual differences are prominent targets for intervention and prevention. Although brain activation has been shown to be associated with individual differences in risk taking, the directionality of the reported brain-behavior associations is less clear. Here, we argue that one aspect contributing to the mixed results is the low convergence between risk-taking measures, especially between the behavioral tasks used to elicit neural functional markers. To address this question, we analyzed within-participant neuroimaging data for two widely used risk-taking tasks collected from the imaging subsample of the Basel-Berlin Risk Study (N = 116 young human adults). Focusing on core brain regions implicated in risk taking (nucleus accumbens, anterior insula, and anterior cingulate cortex), for the two tasks, we examined group-level activation for risky versus safe choices, as well as associations between local functional markers and various risk-related outcomes, including psychometrically derived risk preference factors. While we observed common group-level activation in the two tasks (notably increased nucleus accumbens activation), individual differences analyses support the idea that the presence and directionality of associations between brain activation and risk taking varies as a function of the risk-taking measures used to capture individual differences. Our results have methodological implications for the use of brain markers for intervention or prevention.

Evaluation of the 3D fractal dimension as a marker of structural brain complexity in multiple-acquisition MRI.

Fractal analysis represents a promising new approach to structural neuroimaging data, yet systematic evaluation of the fractal dimension (FD) as a marker of structural brain complexity is scarce. Here we present in-depth methodological assessment of FD estimation in structural brain MRI. On the computational side, we show that spatial scale optimization can significantly improve FD estimation accuracy, as suggested by simulation studies with known FD values. For empirical evaluation, we analyzed two recent open-access neuroimaging data sets (MASSIVE and Midnight Scan Club), stratified by fundamental image characteristics including registration, sequence weighting, spatial resolution, segmentation procedures, tissue type, and image complexity. Deviation analyses showed high repeated-acquisition stability of the FD estimates across both data sets, with differential deviation susceptibility according to image characteristics. While less frequently studied in the literature, FD estimation in T2-weighted images yielded robust outcomes. Importantly, we observed a significant impact of image registration on absolute FD estimates. Applying different registration schemes, we found that unbalanced registration induced (a) repeated-measurement deviation clusters around the registration target, (b) strong bidirectional correlations among image analysis groups, and (c) spurious associations between the FD and an index of structural similarity, and these effects were strongly attenuated by reregistration in both data sets. Indeed, differences in FD between scans did not simply track differences in structure per se, suggesting that structural complexity and structural similarity represent distinct aspects of structural brain MRI. In conclusion, scale optimization can improve FD estimation accuracy, and empirical FD estimates are reliable yet sensitive to image characteristics.

Digital open science-Teaching digital tools for reproducible and transparent research.

An important hallmark of science is the transparency and reproducibility of scientific results. Over the last few years, internet-based technologies have emerged that allow for a representation of the scientific process that goes far beyond traditional methods and analysis descriptions. Using these often freely available tools requires a suite of skills that is not necessarily part of a curriculum in the life sciences. However, funders, journals, and policy makers increasingly require researchers to ensure complete reproducibility of their methods and analyses. To close this gap, we designed an introductory course that guides students towards a reproducible science workflow. Here, we outline the course content and possible extensions, report encountered challenges, and discuss how to integrate such a course in existing curricula.

Variational Bayesian Parameter Estimation Techniques for the General Linear Model.

Variational Bayes (VB), variational maximum likelihood (VML), restricted maximum likelihood (ReML), and maximum likelihood (ML) are cornerstone parametric statistical estimation techniques in the analysis of functional neuroimaging data. However, the theoretical underpinnings of these model parameter estimation techniques are rarely covered in introductory statistical texts. Because of the widespread practical use of VB, VML, ReML, and ML in the neuroimaging community, we reasoned that a theoretical treatment of their relationships and their application in a basic modeling scenario may be helpful for both neuroimaging novices and practitioners alike. In this technical study, we thus revisit the conceptual and formal underpinnings of VB, VML, ReML, and ML and provide a detailed account of their mathematical relationships and implementational details. We further apply VB, VML, ReML, and ML to the general linear model (GLM) with non-spherical error covariance as commonly encountered in the first-level analysis of fMRI data. To this end, we explicitly derive the corresponding free energy objective functions and ensuing iterative algorithms. Finally, in the applied part of our study, we evaluate the parameter and model recovery properties of VB, VML, ReML, and ML, first in an exemplary setting and then in the analysis of experimental fMRI data acquired from a single participant under visual stimulation.

Computing integrated information.

Integrated information theory (IIT) has established itself as one of the leading theories for the study of consciousness. IIT essentially proposes that quantitative consciousness is identical to maximally integrated conceptual information, quantified by a measure called Φmax, and that phenomenological experience corresponds to the associated set of maximally irreducible cause-effect repertoires of a physical system being in a certain state. With the current work, we provide a general formulation of the framework, which comprehensively and parsimoniously expresses Φmax in the language of probabilistic models. Here, the stochastic process describing a system under scrutiny corresponds to a first-order time-invariant Markov process, and all necessary mathematical operations for the definition of Φmax are fully specified by a system's joint probability distribution over two adjacent points in discrete time. We present a detailed constructive rule for the decomposition of a system into two disjoint subsystems based on flexible marginalization and factorization of this joint distribution. Furthermore, we show that for a given joint distribution, virtualization is identical to a flexible factorization enforcing independence between variable subsets. We then validate our formulation in a previously established discrete example system, in which we also illustrate the previously unexplored theoretical issue of quale underdetermination due to non-unique maximally irreducible cause-effect repertoires. Moreover, we show that the current definition of Φ entails its sensitivity to the shape of the conceptual structure in qualia space, thus tying together IIT's measures of quantitative and qualitative consciousness, which we suggest be better disentangled. We propose several modifications of the framework in order to address some of these issues.

Probabilistic delay differential equation modeling of event-related potentials.

"Dynamic causal models" (DCMs) are a promising approach in the analysis of functional neuroimaging data due to their biophysical interpretability and their consolidation of functional-segregative and functional-integrative propositions. In this theoretical note we are concerned with the DCM framework for electroencephalographically recorded event-related potentials (ERP-DCM). Intuitively, ERP-DCM combines deterministic dynamical neural mass models with dipole-based EEG forward models to describe the event-related scalp potential time-series over the entire electrode space. Since its inception, ERP-DCM has been successfully employed to capture the neural underpinnings of a wide range of neurocognitive phenomena. However, in spite of its empirical popularity, the technical literature on ERP-DCM remains somewhat patchy. A number of previous communications have detailed certain aspects of the approach, but no unified and coherent documentation exists. With this technical note, we aim to close this gap and to increase the technical accessibility of ERP-DCM. Specifically, this note makes the following novel contributions: firstly, we provide a unified and coherent review of the mathematical machinery of the latent and forward models constituting ERP-DCM by formulating the approach as a probabilistic latent delay differential equation model. Secondly, we emphasize the probabilistic nature of the model and its variational Bayesian inversion scheme by explicitly deriving the variational free energy function in terms of both the likelihood expectation and variance parameters. Thirdly, we detail and validate the estimation of the model with a special focus on the explicit form of the variational free energy function and introduce a conventional nonlinear optimization scheme for its maximization. Finally, we identify and discuss a number of computational issues which may be addressed in the future development of the approach.

Early haemodynamic changes observed in patients with epilepsy, in a visual experiment and in simulations.

OBJECTIVE: The objective of this study was to investigate whether previously reported early blood oxygen level dependent (BOLD) changes in epilepsy could occur as a result of the modelling techniques rather than physiological changes. METHODS: EEG-fMRI data were analysed from seven patients with focal epilepsy, six control subjects undergoing a visual experiment, in addition to simulations. In six separate analyses the event timing was shifted by either -9,-6,-3,+3,+6 or +9 s relative to the onset of the interictal epileptiform discharge (IED) or stimulus. RESULTS: The visual dataset and simulations demonstrated an overlap between modelled haemodynamic response function (HRF) at event onset and at ± 3 s relative to onset, which diminished at ± 6s. Pre-spike analysis at -6s improved concordance with the assumed IED generating lobe relative to the standard HRF in 43% of patients. CONCLUSION: The visual and simulated dataset findings indicate a form of "temporal bleeding", an overlap between the modelled HRF at time 0 and at ± 3s which attenuated at ± 6s. Pre-spike analysis at -6s may improve concordance. SIGNIFICANCE: This form of analysis should be performed at 6s prior to onset of IED to minimise temporal bleeding effect. The results support the presence of relevant BOLD responses occurring prior to IEDs.

A normative inference approach for optimal sample sizes in decisions from experience.

"Decisions from experience" (DFE) refers to a body of work that emerged in research on behavioral decision making over the last decade. One of the major experimental paradigms employed to study experience-based choice is the "sampling paradigm," which serves as a model of decision making under limited knowledge about the statistical structure of the world. In this paradigm respondents are presented with two payoff distributions, which, in contrast to standard approaches in behavioral economics, are specified not in terms of explicit outcome-probability information, but by the opportunity to sample outcomes from each distribution without economic consequences. Participants are encouraged to explore the distributions until they feel confident enough to decide from which they would prefer to draw from in a final trial involving real monetary payoffs. One commonly employed measure to characterize the behavior of participants in the sampling paradigm is the sample size, that is, the number of outcome draws which participants choose to obtain from each distribution prior to terminating sampling. A natural question that arises in this context concerns the "optimal" sample size, which could be used as a normative benchmark to evaluate human sampling behavior in DFE. In this theoretical study, we relate the DFE sampling paradigm to the classical statistical decision theoretic literature and, under a probabilistic inference assumption, evaluate optimal sample sizes for DFE. In our treatment we go beyond analytically established results by showing how the classical statistical decision theoretic framework can be used to derive optimal sample sizes under arbitrary, but numerically evaluable, constraints. Finally, we critically evaluate the value of deriving optimal sample sizes under this framework as testable predictions for the experimental study of sampling behavior in DFE.

Reliability of information-based integration of EEG and fMRI data: a simulation study.

Most studies involving simultaneous electroencephalographic (EEG) and functional magnetic resonance imaging (fMRI) data rely on the first-order, affine-linear correlation of EEG and fMRI features within the framework of the general linear model. An alternative is the use of information-based measures such as mutual information and entropy, which can also detect higher-order correlations present in the data. The estimate of information-theoretic quantities might be influenced by several parameters, such as the numerosity of the sample, the amount of correlation between variables, and the discretization (or binning) strategy of choice. While these issues have been investigated for invasive neurophysiological data and a number of bias-correction estimates have been developed, there has been no attempt to systematically examine the accuracy of information estimates for the multivariate distributions arising in the context of EEG-fMRI recordings. This is especially important given the differences between electrophysiological and EEG-fMRI recordings. In this study, we drew random samples from simulated bivariate and trivariate distributions, mimicking the statistical properties of EEG-fMRI data. We compared the estimated information shared by simulated random variables with its numerical value and found that the interaction between the binning strategy and the estimation method influences the accuracy of the estimate. Conditional on the simulation assumptions, we found that the equipopulated binning strategy yields the best and most consistent results across distributions and bias correction methods. We also found that within bias correction techniques, the asymptotically debiased (TPMC), the jackknife debiased (JD), and the best upper bound (BUB) approach give similar results, and those are consistent across distributions.

Imaging tactile imagery: changes in brain connectivity support perceptual grounding of mental images in primary sensory cortices.

Constructing mental representations in the absence of sensory stimulation is a fundamental ability of the human mind and has been investigated in numerous brain imaging studies. However, it is still unclear how brain areas facilitating mental construction processes interact with brain regions related to specific sensory representations. In this fMRI study subjects formed mental representations of tactile stimuli either from memory (imagery) or from presentation of actual corresponding vibrotactile patterned stimuli. First our analysis addressed the question of whether tactile imagery recruits primary somatosensory cortex (SI), because the activation of early perceptual areas is classically interpreted as perceptual grounding of the mental image. We also tested whether a network, referred to as 'core construction system', is involved in the generation of mental representations in the somatosensory domain. In fact, we observed imagery-induced activation of SI. We further found support for the notion of a modality independent construction network with the retrosplenial cortices and the precuneus as core components, which were supplemented with the left inferior frontal gyrus (IFG). Finally, psychophysiological interaction (PPI) analyses revealed robust imagery-modulated changes in the connectivity of these construction related areas, which suggests that they orchestrate the assembly of an abstract mental representation. Interestingly, we found increased coupling between prefrontal cortex (left IFG) and SI during mental imagery, indicating the augmentation of an abstract mental representation by reactivating perceptually grounded sensory details.

The structural-functional connectome and the default mode network of the human brain.

An emerging field of human brain imaging deals with the characterization of the connectome, a comprehensive global description of structural and functional connectivity within the human brain. However, the question of how functional and structural connectivity are related has not been fully answered yet. Here, we used different methods to estimate the connectivity between each voxel of the cerebral cortex based on functional magnetic resonance imaging (fMRI) and diffusion tensor imaging (DTI) data in order to obtain observer-independent functional-structural connectomes of the human brain. Probabilistic fiber-tracking and a novel global fiber-tracking technique were used to measure structural connectivity whereas for functional connectivity, full and partial correlations between each voxel pair's fMRI-timecourses were calculated. For every voxel, two vectors consisting of functional and structural connectivity estimates to all other voxels in the cortex were correlated with each other. In this way, voxels structurally and functionally connected to similar regions within the rest of the brain could be identified. Areas forming parts of the 'default mode network' (DMN) showed the highest agreement of structure-function connectivity. Bilateral precuneal and inferior parietal regions were found using all applied techniques, whereas the global tracking algorithm additionally revealed bilateral medial prefrontal cortices and early visual areas. There were no significant differences between the results obtained from full and partial correlations. Our data suggests that the DMN is the functional brain network, which uses the most direct structural connections. Thus, the anatomical profile of the brain seems to shape its functional repertoire and the computation of the whole-brain functional-structural connectome appears to be a valuable method to characterize global brain connectivity within and between populations.

Spontaneous EEG alpha oscillation interacts with positive and negative BOLD responses in the visual-auditory cortices and default-mode network.

The human brain is continually, dynamically active and spontaneous fluctuations in this activity play a functional role in affecting both behavioural and neuronal responses. However, the mechanisms through which this occurs remain poorly understood. Simultaneous EEG-fMRI is a promising technique to study how spontaneous activity modulates the brain's response to stimulation, as temporal indices of ongoing cortical excitability can be integrated with spatially localised evoked responses. Here we demonstrate an interaction between the ongoing power of the electrophysiological alpha oscillation and the magnitude of both positive (PBR) and negative (NBR) fMRI responses to two contrasts of visual checkerboard reversal. Furthermore, the amplitude of pre-stimulus EEG alpha-power significantly modulated the amplitude and shape of subsequent PBR and NBR to the visual stimulus. A nonlinear reduction of visual PBR and an enhancement of auditory NBR and default-mode network NBR were observed in trials preceded by high alpha-power. These modulated areas formed a functionally connected network during a separate resting-state recording. Our findings suggest that the "baseline" state of the brain exhibits considerable trial-to-trial variability which arises from fluctuations in the balance of cortical inhibition/excitation that are represented by respective increases/decreases in the power of the EEG alpha oscillation. The consequence of this spontaneous electrophysiological variability is modulated amplitudes of both PBR and NBR to stimulation. Fluctuations in alpha-power may subserve a functional relationship in the visual-auditory network, acting as mediator for both short and long-range cortical inhibition, the strength of which is represented in part by NBR.

Evidence for neural encoding of Bayesian surprise in human somatosensation.

Accumulating empirical evidence suggests a role of Bayesian inference and learning for shaping neural responses in auditory and visual perception. However, its relevance for somatosensory processing is unclear. In the present study we test the hypothesis that cortical somatosensory processing exhibits dynamics that are consistent with Bayesian accounts of brain function. Specifically, we investigate the cortical encoding of Bayesian surprise, a recently proposed marker of Bayesian perceptual learning, using EEG data recorded from 15 subjects. Capitalizing on a somatosensory mismatch roving paradigm, we performed computational single-trial modeling of evoked somatosensory potentials for the entire peri-stimulus time period in source space. By means of Bayesian model selection, we find that, at 140 ms post-stimulus onset, secondary somatosensory cortex represents Bayesian surprise rather than stimulus change, which is the conventional marker of EEG mismatch responses. In contrast, at 250 ms, right inferior frontal cortex indexes stimulus change. Finally, at 360 ms, our analyses indicate additional perceptual learning attributable to medial cingulate cortex. In summary, the present study provides novel evidence for anatomical-temporal/functional segregation in human somatosensory processing that is consistent with the Bayesian brain hypothesis.

EEG-fMRI based information theoretic characterization of the human perceptual decision system.

The modern metaphor of the brain is that of a dynamic information processing device. In the current study we investigate how a core cognitive network of the human brain, the perceptual decision system, can be characterized regarding its spatiotemporal representation of task-relevant information. We capitalize on a recently developed information theoretic framework for the analysis of simultaneously acquired electroencephalography (EEG) and functional magnetic resonance imaging data (fMRI) (Ostwald et al. (2010), NeuroImage 49: 498-516). We show how this framework naturally extends from previous validations in the sensory to the cognitive domain and how it enables the economic description of neural spatiotemporal information encoding. Specifically, based on simultaneous EEG-fMRI data features from n = 13 observers performing a visual perceptual decision task, we demonstrate how the information theoretic framework is able to reproduce earlier findings on the neurobiological underpinnings of perceptual decisions from the response signal features' marginal distributions. Furthermore, using the joint EEG-fMRI feature distribution, we provide novel evidence for a highly distributed and dynamic encoding of task-relevant information in the human brain.

Multimodal functional network connectivity: an EEG-fMRI fusion in network space.

EEG and fMRI recordings measure the functional activity of multiple coherent networks distributed in the cerebral cortex. Identifying network interaction from the complementary neuroelectric and hemodynamic signals may help to explain the complex relationships between different brain regions. In this paper, multimodal functional network connectivity (mFNC) is proposed for the fusion of EEG and fMRI in network space. First, functional networks (FNs) are extracted using spatial independent component analysis (ICA) in each modality separately. Then the interactions among FNs in each modality are explored by Granger causality analysis (GCA). Finally, fMRI FNs are matched to EEG FNs in the spatial domain using network-based source imaging (NESOI). Investigations of both synthetic and real data demonstrate that mFNC has the potential to reveal the underlying neural networks of each modality separately and in their combination. With mFNC, comprehensive relationships among FNs might be unveiled for the deep exploration of neural activities and metabolic responses in a specific task or neurological state.

Information theoretic approaches to functional neuroimaging.

Information theory is a probabilistic framework that allows the quantification of statistical non-independence between signals of interest. In contrast to other methods used for this purpose, it is model free, i.e., it makes no assumption about the functional form of the statistical dependence or the underlying probability distributions. It thus has the potential to unveil important signal characteristics overlooked by classical data analysis techniques. In this review, we discuss how information theoretic concepts have been applied to the analysis of functional brain imaging data such as functional magnetic resonance imaging and magneto/electroencephalography. We review studies from a number of imaging domains, including the investigation of the brain's functional specialization and integration, neurovascular coupling and multimodal imaging. We demonstrate how information theoretical concepts can be used to answer neurobiological questions and discuss their limitations as well as possible future developments of the framework to advance our understanding of brain function.

The relationship between the visual evoked potential and the gamma band investigated by blind and semi-blind methods.

Gamma Band Activity (GBA) is increasingly studied for its relation with attention, change detection, maintenance of working memory and the processing of sensory stimuli. Activity around the gamma range has also been linked with early visual processing, although the relationship between this activity and the low frequency visual evoked potential (VEP) remains unclear. This study examined the ability of blind and semi-blind source separation techniques to extract sources specifically related to the VEP and GBA in order to shed light on the relationship between them. Blind (Independent Component Analysis-ICA) and semi-Blind (Functional Source Separation-FSS) methods were applied to dense array EEG data recorded during checkerboard stimulation. FSS was performed with both temporal and spectral constraints to identify specifically the generators of the main peak of the VEP (P100) and of the GBA. Source localisation and time-frequency analyses were then used to investigate the properties and co-dependencies between VEP/P100 and GBA. Analysis of the VEP extracted using the different methods demonstrated very similar morphology and localisation of the generators. Single trial time frequency analysis showed higher GBA when a larger amplitude VEP/P100 occurred. Further examination indicated that the evoked (phase-locked) component of the GBA was more related to the P100, whilst the induced component correlated with the VEP as a whole. The results suggest that the VEP and GBA may be generated by the same neuronal populations, and implicate this relationship as a potential mediator of the correlation between the VEP and the Blood Oxygenation Level Dependent (BOLD) effect measured with fMRI.