Data science, human intelligence, and therapeutics discovery: An interview with Sean Escola, Saul Kato, and Pavan Ramkumar.

Sean Escola, Saul Kato, and Pavan Ramkumar explain the importance of data science in their research. They have developed a simple non-parametric statistical method called the Rank-to-Group (RTG) score that identifies hierarchical confounder effects in raw data and machine learning-derived data embeddings. This approach should be generally useful in experiment-analysis cycles and to ensure confounder robustness in machine learning models.

Hierarchical confounder discovery in the experiment-machine learning cycle.

The promise of machine learning (ML) to extract insights from high-dimensional datasets is tempered by confounding variables. It behooves scientists to determine if a model has extracted the desired information or instead fallen prey to bias. Due to features of natural phenomena and experimental design constraints, bioscience datasets are often organized in nested hierarchies that obfuscate the origins of confounding effects and render confounder amelioration methods ineffective. We propose a non-parametric statistical method called the rank-to-group (RTG) score that identifies hierarchical confounder effects in raw data and ML-derived embeddings. We show that RTG scores correctly assign the effects of hierarchical confounders when linear methods fail. In a public biomedical image dataset, we discover unreported effects of experimental design. We then use RTG scores to discover crossmodal correlated variability in a multi-phenotypic biological dataset. This approach should be generally useful in experiment-analysis cycles and to ensure confounder robustness in ML models.

Modern Machine Learning as a Benchmark for Fitting Neural Responses.

Neuroscience has long focused on finding encoding models that effectively ask "what predicts neural spiking?" and generalized linear models (GLMs) are a typical approach. It is often unknown how much of explainable neural activity is captured, or missed, when fitting a model. Here we compared the predictive performance of simple models to three leading machine learning methods: feedforward neural networks, gradient boosted trees (using XGBoost), and stacked ensembles that combine the predictions of several methods. We predicted spike counts in macaque motor (M1) and somatosensory (S1) cortices from standard representations of reaching kinematics, and in rat hippocampal cells from open field location and orientation. Of these methods, XGBoost and the ensemble consistently produced more accurate spike rate predictions and were less sensitive to the preprocessing of features. These methods can thus be applied quickly to detect if feature sets relate to neural activity in a manner not captured by simpler methods. Encoding models built with a machine learning approach accurately predict spike rates and can offer meaningful benchmarks for simpler models.

Population coding of conditional probability distributions in dorsal premotor cortex.

Our bodies and the environment constrain our movements. For example, when our arm is fully outstretched, we cannot extend it further. More generally, the distribution of possible movements is conditioned on the state of our bodies in the environment, which is constantly changing. However, little is known about how the brain represents such distributions, and uses them in movement planning. Here, we record from dorsal premotor cortex (PMd) and primary motor cortex (M1) while monkeys reach to randomly placed targets. The hand's position within the workspace creates probability distributions of possible upcoming targets, which affect movement trajectories and latencies. PMd, but not M1, neurons have increased activity when the monkey's hand position makes it likely the upcoming movement will be in the neurons' preferred directions. Across the population, PMd activity represents probability distributions of individual upcoming reaches, which depend on rapidly changing information about the body's state in the environment.

Premotor and Motor Cortices Encode Reward.

Rewards associated with actions are critical for motivation and learning about the consequences of one's actions on the world. The motor cortices are involved in planning and executing movements, but it is unclear whether they encode reward over and above limb kinematics and dynamics. Here, we report a categorical reward signal in dorsal premotor (PMd) and primary motor (M1) neurons that corresponds to an increase in firing rates when a trial was not rewarded regardless of whether or not a reward was expected. We show that this signal is unrelated to error magnitude, reward prediction error, or other task confounds such as reward consumption, return reach plan, or kinematic differences across rewarded and unrewarded trials. The availability of reward information in motor cortex is crucial for theories of reward-based learning and motivational influences on actions.

Uncertainty leads to persistent effects on reach representations in dorsal premotor cortex.

Every movement we make represents one of many possible actions. In reaching tasks with multiple targets, dorsal premotor cortex (PMd) appears to represent all possible actions simultaneously. However, in many situations we are not presented with explicit choices. Instead, we must estimate the best action based on noisy information and execute it while still uncertain of our choice. Here we asked how both primary motor cortex (M1) and PMd represented reach direction during a task in which a monkey made reaches based on noisy, uncertain target information. We found that with increased uncertainty, neurons in PMd actually enhanced their representation of unlikely movements throughout both planning and execution. The magnitude of this effect was highly variable across sessions, and was correlated with a measure of the monkeys' behavioral uncertainty. These effects were not present in M1. Our findings suggest that PMd represents and maintains a full distribution of potentially correct actions.

Chunking as the result of an efficiency computation trade-off.

How to move efficiently is an optimal control problem, whose computational complexity grows exponentially with the horizon of the planned trajectory. Breaking a compound movement into a series of chunks, each planned over a shorter horizon can thus reduce the overall computational complexity and associated costs while limiting the achievable efficiency. This trade-off suggests a cost-effective learning strategy: to learn new movements we should start with many short chunks (to limit the cost of computation). As practice reduces the impediments to more complex computation, the chunking structure should evolve to allow progressively more efficient movements (to maximize efficiency). Here we show that monkeys learning a reaching sequence over an extended period of time adopt this strategy by performing movements that can be described as locally optimal trajectories. Chunking can thus be understood as a cost-effective strategy for producing and learning efficient movements.

Feature-based attention and spatial selection in frontal eye fields during natural scene search.

When we search for visual objects, the features of those objects bias our attention across the visual landscape (feature-based attention). The brain uses these top-down cues to select eye movement targets (spatial selection). The frontal eye field (FEF) is a prefrontal brain region implicated in selecting eye movements and is thought to reflect feature-based attention and spatial selection. Here, we study how FEF facilitates attention and selection in complex natural scenes. We ask whether FEF neurons facilitate feature-based attention by representing search-relevant visual features or whether they are primarily involved in selecting eye movement targets in space. We show that search-relevant visual features are weakly predictive of gaze in natural scenes and additionally have no significant influence on FEF activity. Instead, FEF activity appears to primarily correlate with the direction of the upcoming eye movement. Our result demonstrates a concrete need for better models of natural scene search and suggests that FEF activity during natural scene search is explained primarily by spatial selection.

Role of expected reward in frontal eye field during natural scene search.

When a saccade is expected to result in a reward, both neural activity in oculomotor areas and the saccade itself (e.g., its vigor and latency) are altered (compared with when no reward is expected). As such, it is unclear whether the correlations of neural activity with reward indicate a representation of reward beyond a movement representation; the modulated neural activity may simply represent the differences in motor output due to expected reward. Here, to distinguish between these possibilities, we trained monkeys to perform a natural scene search task while we recorded from the frontal eye field (FEF). Indeed, when reward was expected (i.e., saccades to the target), FEF neurons showed enhanced responses. Moreover, when monkeys accidentally made eye movements to the target, firing rates were lower than when they purposively moved to the target. Thus, neurons were modulated by expected reward rather than simply the presence of the target. We then fit a model that simultaneously included components related to expected reward and saccade parameters. While expected reward led to shorter latency and higher velocity saccades, these behavioral changes could not fully explain the increased FEF firing rates. Thus, FEF neurons appear to encode motivational factors such as reward expectation, above and beyond the kinematic and behavioral consequences of imminent reward.

Visual information representation and rapid-scene categorization are simultaneous across cortex: An MEG study.

Perceiving the visual world around us requires the brain to represent the features of stimuli and to categorize the stimulus based on these features. Incorrect categorization can result either from errors in visual representation or from errors in processes that lead to categorical choice. To understand the temporal relationship between the neural signatures of such systematic errors, we recorded whole-scalp magnetoencephalography (MEG) data from human subjects performing a rapid-scene categorization task. We built scene category decoders based on (1) spatiotemporally resolved neural activity, (2) spatial envelope (SpEn) image features, and (3) behavioral responses. Using confusion matrices, we tracked how well the pattern of errors from neural decoders could be explained by SpEn decoders and behavioral errors, over time and across cortical areas. Across the visual cortex and the medial temporal lobe, we found that both SpEn and behavioral errors explained unique variance in the errors of neural decoders. Critically, these effects were nearly simultaneous, and most prominent between 100 and 250ms after stimulus onset. Thus, during rapid-scene categorization, neural processes that ultimately result in behavioral categorization are simultaneous and co-localized with neural processes underlying visual information representation.

Modeling peripheral visual acuity enables discovery of gaze strategies at multiple time scales during natural scene search.

Like humans, monkeys make saccades nearly three times a second. To understand the factors guiding this frequent decision, computational models of vision attempt to predict fixation locations using bottom-up visual features and top-down goals. How do the relative influences of these factors evolve over multiple time scales? Here we analyzed visual features at fixations using a retinal transform that provides realistic visual acuity by suitably degrading visual information in the periphery. In a task in which monkeys searched for a Gabor target in natural scenes, we characterized the relative importance of bottom-up and task-relevant influences by decoding fixated from nonfixated image patches based on visual features. At fast time scales, we found that search strategies can vary over the course of a single trial, with locations of higher saliency, target-similarity, edge­energy, and orientedness looked at later on in the trial. At slow time scales, we found that search strategies can be refined over several weeks of practice, and the influence of target orientation was significant only in the latter of two search tasks. Critically, these results were not observed without applying the retinal transform. Our results suggest that saccade-guidance strategies become apparent only when models take into account degraded visual representation in the periphery.

Group-level spatial independent component analysis of Fourier envelopes of resting-state MEG data.

We developed a data-driven method to spatiotemporally and spectrally characterize the dynamics of brain oscillations in resting-state magnetoencephalography (MEG) data. The method, called envelope spatial Fourier independent component analysis (eSFICA), maximizes the spatial and spectral sparseness of Fourier energies of a cortically constrained source current estimate. We compared this method using a simulated data set against 5 other variants of independent component analysis and found that eSFICA performed on par with its temporal variant, eTFICA, and better than other ICA variants, in characterizing dynamics at time scales of the order of minutes. We then applied eSFICA to real MEG data obtained from 9 subjects during rest. The method identified several networks showing within- and cross-frequency inter-areal functional connectivity profiles which resemble previously reported resting-state networks, such as the bilateral sensorimotor network at ~20Hz, the lateral and medial parieto-occipital sources at ~10Hz, a subset of the default-mode network at ~8 and ~15Hz, and lateralized temporal lobe sources at ~8Hz. Finally, we interpreted the estimated networks as spatiospectral filters and applied the filters to obtain the dynamics during a natural stimulus sequence presented to the same 9 subjects. We observed occipital alpha modulation to visual stimuli, bilateral rolandic mu modulation to tactile stimuli and video clips of hands, and the temporal lobe network modulation to speech stimuli, but no modulation of the sources in the default-mode network. We conclude that (1) the proposed method robustly detects inter-areal cross-frequency networks at long time scales, (2) the functional relevance of the resting-state networks can be probed by applying the obtained spatiospectral filters to data from measurements with controlled external stimulation.

Feature-specific information processing precedes concerted activation in human visual cortex.

Current knowledge about the precise timing of visual input to the cortex relies largely on spike timings in monkeys and evoked-response latencies in humans. However, quantifying the activation onset does not unambiguously describe the timing of stimulus-feature-specific information processing. Here, we investigated the information content of the early human visual cortical activity by decoding low-level visual features from single-trial magnetoencephalographic (MEG) responses. MEG was measured from nine healthy subjects as they viewed annular sinusoidal gratings (spanning the visual field from 2 to 10° for a duration of 1 s), characterized by spatial frequency (0.33 cycles/degree or 1.33 cycles/degree) and orientation (45° or 135°); gratings were either static or rotated clockwise or anticlockwise from 0 to 180°. Time-resolved classifiers using a 20 ms moving window exceeded chance level at 51 ms (the later edge of the window) for spatial frequency, 65 ms for orientation, and 98 ms for rotation direction. Decoding accuracies of spatial frequency and orientation peaked at 70 and 90 ms, respectively, coinciding with the peaks of the onset evoked responses. Within-subject time-insensitive pattern classifiers decoded spatial frequency and orientation simultaneously (mean accuracy 64%, chance 25%) and rotation direction (mean 82%, chance 50%). Classifiers trained on data from other subjects decoded the spatial frequency (73%), but not the orientation, nor the rotation direction. Our results indicate that unaveraged brain responses contain decodable information about low-level visual features already at the time of the earliest cortical evoked responses, and that representations of spatial frequency are highly robust across individuals.

Testing independent component patterns by inter-subject or inter-session consistency.

Independent component analysis (ICA) is increasingly used to analyze patterns of spontaneous activity in brain imaging. However, there are hardly any methods for answering the fundamental question: are the obtained components statistically significant? Most methods considering the significance of components either consider group-differences or use arbitrary thresholds with weak statistical justification. In previous work, we proposed a statistically principled method for testing if the coefficients in the mixing matrix are similar in different subjects or sessions. In many applications of ICA, however, we would like to test the reliability of the independent components themselves and not the mixing coefficients. Here, we develop a test for such an inter-subject consistency by extending our previous theory. The test is applicable, for example, to the spatial activity patterns obtained by spatial ICA in resting-state fMRI. We further improve both this and the previously proposed testing method by introducing a new way of correcting for multiple testing, new variants of the clustering method, and a computational approximation which greatly reduces the memory and computation required.

Characterization of neuromagnetic brain rhythms over time scales of minutes using spatial independent component analysis.

Independent component analysis (ICA) of electroencephalographic (EEG) and magnetoencephalographic (MEG) data is usually performed over the temporal dimension: each channel is one row of the data matrix, and a linear transformation maximizing the independence of component time courses is sought. In functional magnetic resonance imaging (fMRI), by contrast, most studies use spatial ICA: each time point constitutes a row of the data matrix, and independence of the spatial patterns is maximized. Here, we show the utility of spatial ICA in characterizing oscillatory neuromagnetic signals. We project the sensor data into cortical space using a standard minimum-norm estimate and apply a sparsifying transform to focus on oscillatory signals. The resulting method, spatial Fourier-ICA, provides a concise summary of the spatiotemporal and spectral content of spontaneous neuromagnetic oscillations in cortical source space over time scales of minutes. Spatial Fourier-ICA applied to resting-state and naturalistic stimulation MEG data from nine healthy subjects revealed consistent components covering the early visual, somatosensory and motor cortices with spectral peaks at ∼10 and ∼20 Hz. The proposed method seems valuable for inferring functional connectivity, stimulus-related modulation of rhythmic activity, and their commonalities across subjects from nonaveraged MEG data.

Aberrant temporal and spatial brain activity during rest in patients with chronic pain.

In the absence of external stimuli, human hemodynamic brain activity displays slow intrinsic variations. To find out whether such fluctuations would be altered by persistent pain, we asked 10 patients with unrelenting chronic pain of different etiologies and 10 sex- and age-matched control subjects to rest with eyes open during 3-T functional MRI. Independent component analysis was used to identify functionally coupled brain networks. Time courses of an independent component comprising the insular cortices of both hemispheres showed stronger spectral power at 0.12 to 0.25 Hz in patients than in control subjects, with the largest difference at 0.16 Hz. A similar but weaker effect was seen in the anterior cingulate cortex, whereas activity of the precuneus and early visual cortex, used as a control site, did not differ between the groups. In the patient group, seed point-based correlation analysis revealed altered spatial connectivity between insulae and anterior cingulate cortex. The results imply both temporally and spatially aberrant activity of the affective pain-processing areas in patients suffering from chronic pain. The accentuated 0.12- to 0.25-Hz fluctuations in the patient group might be related to altered activity of the autonomic nervous system.

Oscillatory response function: towards a parametric model of rhythmic brain activity.

Rhythmic brain activity, measured by magnetoencephalography (MEG), is modulated during stimulation and task performance. Here, we introduce an oscillatory response function (ORF) to predict the dynamic suppression-rebound modulation of brain rhythms during a stimulus sequence. We derived a class of parametric models for the ORF in a generalized convolution framework. The model parameters were estimated from MEG data acquired from 10 subjects during bilateral tactile stimulation of fingers (stimulus rates of 4 Hz and 10 Hz in blocks of 0.5, 1, 2, and 4 s). The envelopes of the 17-23 Hz rhythmic activity, computed for sensors above the rolandic region, correlated 25%-43% better with the envelopes predicted by the models than by the stimulus time course (boxcar). A linear model with separate convolution kernels for onset and offset responses gave the best prediction. We studied the generalizability of this model with data from 5 different subjects during a separate bilateral tactile sequence by first identifying neural sources of the 17-23 Hz activity using cortically constrained minimum norm estimates. Both the model and the boxcar predicted strongest modulation in the primary motor cortex. For short-duration stimulus blocks, the model predicted the envelope of the cortical currents 20% better than the boxcar did. These results suggest that ORFs could concisely describe brain rhythms during different stimuli, tasks, and pathologies.

Independent component analysis of short-time Fourier transforms for spontaneous EEG/MEG analysis.

Analysis of spontaneous EEG/MEG needs unsupervised learning methods. While independent component analysis (ICA) has been successfully applied on spontaneous fMRI, it seems to be too sensitive to technical artifacts in EEG/MEG. We propose to apply ICA on short-time Fourier transforms of EEG/MEG signals, in order to find more "interesting" sources than with time-domain ICA, and to more meaningfully sort the obtained components. The method is especially useful for finding sources of rhythmic activity. Furthermore, we propose to use a complex mixing matrix to model sources which are spatially extended and have different phases in different EEG/MEG channels. Simulations with artificial data and experiments on resting-state MEG demonstrate the utility of the method.