A drift diffusion model analysis of age-related impact on multisensory decision-making processes.

Older adults (OAs) are typically slower and/or less accurate in forming perceptual choices relative to younger adults. Despite perceptual deficits, OAs gain from integrating information across senses, yielding multisensory benefits. However, the cognitive processes underlying these seemingly discrepant ageing effects remain unclear. To address this knowledge gap, 212 participants (18-90 years old) performed an online object categorisation paradigm, whereby age-related differences in Reaction Times (RTs) and choice accuracy between audiovisual (AV), visual (V), and auditory (A) conditions could be assessed. Whereas OAs were slower and less accurate across sensory conditions, they exhibited greater RT decreases between AV and V conditions, showing a larger multisensory benefit towards decisional speed. Hierarchical Drift Diffusion Modelling (HDDM) was fitted to participants' behaviour to probe age-related impacts on the latent multisensory decision formation processes. For OAs, HDDM demonstrated slower evidence accumulation rates across sensory conditions coupled with increased response caution for AV trials of higher difficulty. Notably, for trials of lower difficulty we found multisensory benefits in evidence accumulation that increased with age, but not for trials of higher difficulty, in which increased response caution was instead evident. Together, our findings reconcile age-related impacts on multisensory decision-making, indicating greater multisensory evidence accumulation benefits with age underlying enhanced decisional speed.

A tensor decomposition reveals ageing-induced differences in muscle and grip-load force couplings during object lifting.

Do motor patterns of object lifting movements change as a result of ageing? Here we propose a methodology for the characterization of these motor patterns across individuals of different age groups. Specifically, we employ a bimanual grasp-lift-replace protocol with younger and older adults and combine measurements of muscle activity with grip and load forces to provide a window into the motor strategies supporting effective object lifts. We introduce a tensor decomposition to identify patterns of muscle activity and grip-load force ratios while also characterizing their temporal profiles and relative activation across object weights and participants of different age groups. We then probe age-induced changes in these components. A classification analysis reveals three motor components that are differentially recruited between the two age groups. Linear regression analyses further show that advanced age and poorer manual dexterity can be predicted by the coupled activation of forearm and hand muscles which is associated with high levels of grip force. Our findings suggest that ageing may induce stronger muscle couplings in distal aspects of the upper limbs, and a less economic grasping strategy to overcome age-related decline in manual dexterity.

A systematic review of quantitative EEG findings in Fibromyalgia, Chronic Fatigue Syndrome and Long COVID.

Fibromyalgia Syndrome (FMS), Myalgic Encephalomyelitis/Chronic Fatigue Syndrome (ME/CFS) and Long COVID (LC) are similar multisymptom clinical syndromes but with difference in dominant symptoms in each individual. There is existing and emerging literature on possible functional alterations of the central nervous system in these conditions. This review aims to synthesise and appraise the literature on resting-state quantitative EEG (qEEG) in FMS, ME/CFS and LC, drawing on previous research on FMS and ME/CFS to help understand neuropathophysiology of the new condition LC. A systematic search of MEDLINE, Embase, CINHAL, PsycINFO and Web of Science databases for articles published between December 1994 and September 2023 was performed. Out of the initial 2510 studies identified, 17 articles were retrieved that met all the predetermined selection criteria, particularly of assessing qEEG changes in one of the three conditions compared to healthy controls. All studies scored moderate to high quality on the Newcastle-Ottawa scale. There was a general trend for decreased low-frequency EEG band activity (delta, theta, and alpha) and increased high-frequency EEG beta activity in FMS, differing to that found in ME/CFS. The limited LC studies included in this review focused mainly on cognitive impairments and showed mixed findings not consistent with patterns observed in FMS and ME/CFS. Our findings suggest different patterns of qEEG brainwave activity in FMS and ME/CFS. Further research is required to explore whether there are phenotypes within LC that have EEG signatures similar to FMS or ME/CFS. This could inform identification of reliable diagnostic markers and possible targets for neuromodulation therapies tailored to each clinical syndrome.

Dissecting muscle synergies in the task space.

The muscle synergy is a guiding concept in motor control research that relies on the general notion of muscles 'working together' towards task performance. However, although the synergy concept has provided valuable insights into motor coordination, muscle interactions have not been fully characterised with respect to task performance. Here, we address this research gap by proposing a novel perspective to the muscle synergy that assigns specific functional roles to muscle couplings by characterising their task-relevance. Our novel perspective provides nuance to the muscle synergy concept, demonstrating how muscular interactions can 'work together' in different ways: (1) irrespective of the task at hand but also (2) redundantly or (3) complementarily towards common task-goals. To establish this perspective, we leverage information- and network-theory and dimensionality reduction methods to include discrete and continuous task parameters directly during muscle synergy extraction. Specifically, we introduce co-information as a measure of the task-relevance of muscle interactions and use it to categorise such interactions as task-irrelevant (present across tasks), redundant (shared task information), or synergistic (different task information). To demonstrate these types of interactions in real data, we firstly apply the framework in a simple way, revealing its added functional and physiological relevance with respect to current approaches. We then apply the framework to large-scale datasets and extract generalizable and scale-invariant representations consisting of subnetworks of synchronised muscle couplings and distinct temporal patterns. The representations effectively capture the functional interplay between task end-goals and biomechanical affordances and the concurrent processing of functionally similar and complementary task information. The proposed framework unifies the capabilities of current approaches in capturing distinct motor features while providing novel insights and research opportunities through a nuanced perspective to the muscle synergy.

Speech perception difficulty modulates theta-band encoding of articulatory synergies.

The human brain tracks available speech acoustics and extrapolates missing information such as the speaker's articulatory patterns. However, the extent to which articulatory reconstruction supports speech perception remains unclear. This study explores the relationship between articulatory reconstruction and task difficulty. Participants listened to sentences and performed a speech-rhyming task. Real kinematic data of the speaker's vocal tract were recorded via electromagnetic articulography (EMA) and aligned to corresponding acoustic outputs. We extracted articulatory synergies from the EMA data with principal component analysis (PCA) and employed partial information decomposition (PID) to separate the electroencephalographic (EEG) encoding of acoustic and articulatory features into unique, redundant, and synergistic atoms of information. We median-split sentences into easy (ES) and hard (HS) based on participants' performance and found that greater task difficulty involved greater encoding of unique articulatory information in the theta band. We conclude that fine-grained articulatory reconstruction plays a complementary role in the encoding of speech acoustics, lending further support to the claim that motor processes support speech perception.NEW & NOTEWORTHY Top-down processes originating from the motor system contribute to speech perception through the reconstruction of the speaker's articulatory movement. This study investigates the role of such articulatory simulation under variable task difficulty. We show that more challenging listening tasks lead to increased encoding of articulatory kinematics in the theta band and suggest that, in such situations, fine-grained articulatory reconstruction complements acoustic encoding.

An emergent temporal basis set robustly supports cerebellar time-series learning.

The cerebellum is considered a "learning machine" essential for time interval estimation underlying motor coordination and other behaviors. Theoretical work has proposed that the cerebellum's input recipient structure, the granule cell layer (GCL), performs pattern separation of inputs that facilitates learning in Purkinje cells (P-cells). However, the relationship between input reformatting and learning has remained debated, with roles emphasized for pattern separation features from sparsification to decorrelation. We took a novel approach by training a minimalist model of the cerebellar cortex to learn complex time-series data from time-varying inputs, typical during movements. The model robustly produced temporal basis sets from these inputs, and the resultant GCL output supported better learning of temporally complex target functions than mossy fibers alone. Learning was optimized at intermediate threshold levels, supporting relatively dense granule cell activity, yet the key statistical features in GCL population activity that drove learning differed from those seen previously for classification tasks. These findings advance testable hypotheses for mechanisms of temporal basis set formation and predict that moderately dense population activity optimizes learning.NEW & NOTEWORTHY During movement, mossy fiber inputs to the cerebellum relay time-varying information with strong intrinsic relationships to ongoing movement. Are such mossy fibers signals sufficient to support Purkinje signals and learning? In a model, we show how the GCL greatly improves Purkinje learning of complex, temporally dynamic signals relative to mossy fibers alone. Learning-optimized GCL population activity was moderately dense, which retained intrinsic input variance while also performing pattern separation.

Corrigendum: ACTIVATE-2: A double-blind randomized trial of BCG vaccination against COVID-19 in individuals at risk.

[This corrects the article DOI: 10.3389/fimmu.2022.873067.].

ACTIVATE-2: A Double-Blind Randomized Trial of BCG Vaccination Against COVID-19 in Individuals at Risk.

In a recent study of our group with the acronym ACTIVATE, Bacillus Calmete-Guérin (BCG) vaccination reduced the occurrence of new infections compared to placebo vaccination in the elderly. Most benefit was found for respiratory infections. The ACTIVATE-2 study was launched to assess the efficacy of BCG vaccination against coronavirus disease 2019 (COVID-19). In this multicenter, double-blind trial, 301 volunteers aged 50 years or older were randomized (1:1) to be vaccinated with BCG or placebo. The trial end points were the incidence of COVID-19 and the presence of anti-severe acute respiratory syndrome coronavirus 2 (anti-SARS-CoV-2) antibodies, which were both evaluated through 6 months after study intervention. Results revealed 68% relative reduction of the risk to develop COVID-19, using clinical criteria or/and laboratory diagnosis, in the group of BCG vaccine recipients compared with placebo-vaccinated controls, during a 6-month follow-up (OR 0.32, 95% CI 0.13-0.79). In total, eight patients were in need of hospitalization for COVID-19: six in the placebo group and two in the BCG group. Three months after study intervention, positive anti-SARS-CoV-2 antibodies were noted in 1.3% of volunteers in the placebo group and in 4.7% of participants in BCG-vaccinated group. These data indicate that BCG vaccination confers some protection against possible COVID-19 among patients older than 50 years with comorbidities. BCG vaccination may be a promising approach against the COVID-19 pandemic.

Clinical Characteristics and Mechanisms of Musculoskeletal Pain in Long COVID.

Objective: Musculoskeletal (MSK) pain is being increasingly reported by patients as one of the most common persistent symptoms in post-COVID-19 syndrome or Long COVID. However, there is a lack of understanding of its prevalence, characteristics, and underlying pathophysiological mechanisms. The objective of this review is to identify and describe the features and characteristics of MSK pain in Long COVID patients. Methods: The narrative review involved a literature search of the following online databases: MEDLINE (OVID), EMBASE (OVID), CINAHL, PsyclNFO, and Web of Science (December 2019 to February 2022). We included observational studies that investigated the prevalence, characteristics, risk factors and mechanisms of MSK pain in Long COVID. After screening and reviewing the initial literature search results, a total of 35 studies were included in this review. Results: The overall reported prevalence of MSK pain in Long COVID ranged widely from 0.3% to 65.2%. The pain has been reported to be localized to a particular region or generalized and widespread. No consistent pattern of progression of MSK pain symptoms over time was identified. Female gender and higher BMI could be potential risk factors for Long COVID MSK pain, but no clear association has been found with age and ethnicity. Different pathophysiological mechanisms have been hypothesized to contribute to MSK pain in Long COVID including increased production of proinflammatory cytokines, immune cell hyperactivation, direct viral entry of neurological and MSK system cells, and psychological factors. Conclusion: MSK pain is one of the most common symptoms in Long COVID. Most of the current literature on Long COVID focuses on reporting the prevalence of persistent MSK pain. Studies describing the pain characteristics are scarce. The precise mechanism of MSK pain in Long COVID is yet to be investigated. Future research must explore the characteristics, risk factors, natural progression, and underlying mechanisms of MSK pain in Long COVID.

A network information theoretic framework to characterise muscle synergies in space and time.

Objective. Current approaches to muscle synergy extraction rely on linear dimensionality reduction algorithms that make specific assumptions on the underlying signals. However, to capture nonlinear time varying, large-scale but also muscle-specific interactions, a more generalised approach is required.Approach. Here we developed a novel framework for muscle synergy extraction that relaxes model assumptions by using a combination of information and network theory and dimensionality reduction. We first quantify informational dynamics between muscles, time-samples or muscle-time pairings using a novel mutual information formulation. We then model these pairwise interactions as multiplex networks and identify modules representing the network architecture. We employ this modularity criterion as the input parameter for dimensionality reduction, which verifiably extracts the identified modules, and also to characterise salient structures within each module.Main results. This novel framework captures spatial, temporal and spatiotemporal interactions across two benchmark datasets of reaching movements, producing distinct spatial groupings and both tonic and phasic temporal patterns. Readily interpretable muscle synergies spanning multiple spatial and temporal scales were identified, demonstrating significant task dependence, ability to capture trial-to-trial fluctuations and concordance across participants. Furthermore, our framework identifies submodular structures that represent the distributed networks of co-occurring signal interactions across scales.Significance. The capabilities of this framework are illustrated through the concomitant continuity with previous research and novelty of the insights gained. Several previous limitations are circumvented including the extraction of functionally meaningful and multiplexed pairwise muscle couplings under relaxed model assumptions. The extracted synergies provide a holistic view of the movement while important details of task performance are readily interpretable. The identified muscle groupings transcend biomechanical constraints and the temporal patterns reveal characteristics of fundamental motor control mechanisms. We conclude that this framework opens new opportunities for muscle synergy research and can constitute a bridge between existing models and recent network-theoretic endeavours.

Neural Encoding of Active Multi-Sensing Enhances Perceptual Decision-Making via a Synergistic Cross-Modal Interaction.

Most perceptual decisions rely on the active acquisition of evidence from the environment involving stimulation from multiple senses. However, our understanding of the neural mechanisms underlying this process is limited. Crucially, it remains elusive how different sensory representations interact in the formation of perceptual decisions. To answer these questions, we used an active sensing paradigm coupled with neuroimaging, multivariate analysis, and computational modeling to probe how the human brain processes multisensory information to make perceptual judgments. Participants of both sexes actively sensed to discriminate two texture stimuli using visual (V) or haptic (H) information or the two sensory cues together (VH). Crucially, information acquisition was under the participants' control, who could choose where to sample information from and for how long on each trial. To understand the neural underpinnings of this process, we first characterized where and when active sensory experience (movement patterns) is encoded in human brain activity (EEG) in the three sensory conditions. Then, to offer a neurocomputational account of active multisensory decision formation, we used these neural representations of active sensing to inform a drift diffusion model of decision-making behavior. This revealed a multisensory enhancement of the neural representation of active sensing, which led to faster and more accurate multisensory decisions. We then dissected the interactions between the V, H, and VH representations using a novel information-theoretic methodology. Ultimately, we identified a synergistic neural interaction between the two unisensory (V, H) representations over contralateral somatosensory and motor locations that predicted multisensory (VH) decision-making performance.SIGNIFICANCE STATEMENT In real-world settings, perceptual decisions are made during active behaviors, such as crossing the road on a rainy night, and include information from different senses (e.g., car lights, slippery ground). Critically, it remains largely unknown how sensory evidence is combined and translated into perceptual decisions in such active scenarios. Here we address this knowledge gap. First, we show that the simultaneous exploration of information across senses (multi-sensing) enhances the neural encoding of active sensing movements. Second, the neural representation of active sensing modulates the evidence available for decision; and importantly, multi-sensing yields faster evidence accumulation. Finally, we identify a cross-modal interaction in the human brain that correlates with multisensory performance, constituting a putative neural mechanism for forging active multisensory perception.

Neurocomputational mechanisms underlying cross-modal associations and their influence on perceptual decisions.

When exposed to complementary features of information across sensory modalities, our brains formulate cross-modal associations between features of stimuli presented separately to multiple modalities. For example, auditory pitch-visual size associations map high-pitch tones with small-size visual objects, and low-pitch tones with large-size visual objects. Preferential, or congruent, cross-modal associations have been shown to affect behavioural performance, i.e. choice accuracy and reaction time (RT) across multisensory decision-making paradigms. However, the neural mechanisms underpinning such influences in perceptual decision formation remain unclear. Here, we sought to identify when perceptual improvements from associative congruency emerge in the brain during decision formation. In particular, we asked whether such improvements represent 'early' sensory processing benefits, or 'late' post-sensory changes in decision dynamics. Using a modified version of the Implicit Association Test (IAT), coupled with electroencephalography (EEG), we measured the neural activity underlying the effect of auditory stimulus-driven pitch-size associations on perceptual decision formation. Behavioural results showed that participants responded significantly faster during trials when auditory pitch was congruent, rather than incongruent, with its associative visual size counterpart. We used multivariate Linear Discriminant Analysis (LDA) to characterise the spatiotemporal dynamics of EEG activity underpinning IAT performance. We found an 'Early' component (∼100-110 ms post-stimulus onset) coinciding with the time of maximal discrimination of the auditory stimuli), and a 'Late' component (∼330-340 ms post-stimulus onset) underlying IAT performance. To characterise the functional role of these components in decision formation, we incorporated a neurally-informed Hierarchical Drift Diffusion Model (HDDM), revealing that the Late component decreases response caution, requiring less sensory evidence to be accumulated, whereas the Early component increased the duration of sensory-encoding processes for incongruent trials. Overall, our results provide a mechanistic insight into the contribution of 'early' sensory processing, as well as 'late' post-sensory neural representations of associative congruency to perceptual decision formation.

Auditory information enhances post-sensory visual evidence during rapid multisensory decision-making.

Despite recent progress in understanding multisensory decision-making, a conclusive mechanistic account of how the brain translates the relevant evidence into a decision is lacking. Specifically, it remains unclear whether perceptual improvements during rapid multisensory decisions are best explained by sensory (i.e., 'Early') processing benefits or post-sensory (i.e., 'Late') changes in decision dynamics. Here, we employ a well-established visual object categorisation task in which early sensory and post-sensory decision evidence can be dissociated using multivariate pattern analysis of the electroencephalogram (EEG). We capitalize on these distinct neural components to identify when and how complementary auditory information influences the encoding of decision-relevant visual evidence in a multisensory context. We show that it is primarily the post-sensory, rather than the early sensory, EEG component amplitudes that are being amplified during rapid audiovisual decision-making. Using a neurally informed drift diffusion model we demonstrate that a multisensory behavioral improvement in accuracy arises from an enhanced quality of the relevant decision evidence, as captured by the post-sensory EEG component, consistent with the emergence of multisensory evidence in higher-order brain areas.

Deciphering the functional role of spatial and temporal muscle synergies in whole-body movements.

Voluntary movement is hypothesized to rely on a limited number of muscle synergies, the recruitment of which translates task goals into effective muscle activity. In this study, we investigated how to analytically characterize the functional role of different types of muscle synergies in task performance. To this end, we recorded a comprehensive dataset of muscle activity during a variety of whole-body pointing movements. We decomposed the electromyographic (EMG) signals using a space-by-time modularity model which encompasses the main types of synergies. We then used a task decoding and information theoretic analysis to probe the role of each synergy by mapping it to specific task features. We found that the temporal and spatial aspects of the movements were encoded by different temporal and spatial muscle synergies, respectively, consistent with the intuition that there should a correspondence between major attributes of movement and major features of synergies. This approach led to the development of a novel computational method for comparing muscle synergies from different participants according to their functional role. This functional similarity analysis yielded a small set of temporal and spatial synergies that describes the main features of whole-body reaching movements.

Space-by-Time Modular Decomposition Effectively Describes Whole-Body Muscle Activity During Upright Reaching in Various Directions.

The modular control hypothesis suggests that motor commands are built from precoded modules whose specific combined recruitment can allow the performance of virtually any motor task. Despite considerable experimental support, this hypothesis remains tentative as classical findings of reduced dimensionality in muscle activity may also result from other constraints (biomechanical couplings, data averaging or low dimensionality of motor tasks). Here we assessed the effectiveness of modularity in describing muscle activity in a comprehensive experiment comprising 72 distinct point-to-point whole-body movements during which the activity of 30 muscles was recorded. To identify invariant modules of a temporal and spatial nature, we used a space-by-time decomposition of muscle activity that has been shown to encompass classical modularity models. To examine the decompositions, we focused not only on the amount of variance they explained but also on whether the task performed on each trial could be decoded from the single-trial activations of modules. For the sake of comparison, we confronted these scores to the scores obtained from alternative non-modular descriptions of the muscle data. We found that the space-by-time decomposition was effective in terms of data approximation and task discrimination at comparable reduction of dimensionality. These findings show that few spatial and temporal modules give a compact yet approximate representation of muscle patterns carrying nearly all task-relevant information for a variety of whole-body reaching movements.

Correlation of neural activity with behavioral kinematics reveals distinct sensory encoding and evidence accumulation processes during active tactile sensing.

Many real-world decisions rely on active sensing, a dynamic process for directing our sensors (e.g. eyes or fingers) across a stimulus to maximize information gain. Though ecologically pervasive, limited work has focused on identifying neural correlates of the active sensing process. In tactile perception, we often make decisions about an object/surface by actively exploring its shape/texture. Here we investigate the neural correlates of active tactile decision-making by simultaneously measuring electroencephalography (EEG) and finger kinematics while subjects interrogated a haptic surface to make perceptual judgments. Since sensorimotor behavior underlies decision formation in active sensing tasks, we hypothesized that the neural correlates of decision-related processes would be detectable by relating active sensing to neural activity. Novel brain-behavior correlation analysis revealed that three distinct EEG components, localizing to right-lateralized occipital cortex (LOC), middle frontal gyrus (MFG), and supplementary motor area (SMA), respectively, were coupled with active sensing as their activity significantly correlated with finger kinematics. To probe the functional role of these components, we fit their single-trial-couplings to decision-making performance using a hierarchical-drift-diffusion-model (HDDM), revealing that the LOC modulated the encoding of the tactile stimulus whereas the MFG predicted the rate of information integration towards a choice. Interestingly, the MFG disappeared from components uncovered from control subjects performing active sensing but not required to make perceptual decisions. By uncovering the neural correlates of distinct stimulus encoding and evidence accumulation processes, this study delineated, for the first time, the functional role of cortical areas in active tactile decision-making.

Biofeedback Signals for Robotic Rehabilitation: Assessment of Wrist Muscle Activation Patterns in Healthy Humans.

Electrophysiological recordings from human muscles can serve as control signals for robotic rehabilitation devices. Given that many diseases affecting the human sensorimotor system are associated with abnormal patterns of muscle activation, such biofeedback can optimize human-robot interaction and ultimately enhance motor recovery. To understand how mechanical constraints and forces imposed by a robot affect muscle synergies, we mapped the muscle activity of seven major arm muscles in healthy individuals performing goal-directed discrete wrist movements constrained by a wrist robot. We tested six movement directions and four force conditions typically experienced during robotic rehabilitation. We analyzed electromyographic (EMG) signals using a space-by-time decomposition and we identified a set of spatial and temporal modules that compactly described the EMG activity and were robust across subjects. For each trial, coefficients expressing the strength of each combination of modules and representing the underlying muscle recruitment, allowed for a highly reliable decoding of all experimental conditions. The decomposition provides compact representations of the observable muscle activation constrained by a robotic device. Results indicate that a low-dimensional control scheme incorporating EMG biofeedback could be an effective add-on for robotic rehabilitative protocols seeking to improve impaired motor function in humans.

Using Matrix and Tensor Factorizations for the Single-Trial Analysis of Population Spike Trains.

Advances in neuronal recording techniques are leading to ever larger numbers of simultaneously monitored neurons. This poses the important analytical challenge of how to capture compactly all sensory information that neural population codes carry in their spatial dimension (differences in stimulus tuning across neurons at different locations), in their temporal dimension (temporal neural response variations), or in their combination (temporally coordinated neural population firing). Here we investigate the utility of tensor factorizations of population spike trains along space and time. These factorizations decompose a dataset of single-trial population spike trains into spatial firing patterns (combinations of neurons firing together), temporal firing patterns (temporal activation of these groups of neurons) and trial-dependent activation coefficients (strength of recruitment of such neural patterns on each trial). We validated various factorization methods on simulated data and on populations of ganglion cells simultaneously recorded in the salamander retina. We found that single-trial tensor space-by-time decompositions provided low-dimensional data-robust representations of spike trains that capture efficiently both their spatial and temporal information about sensory stimuli. Tensor decompositions with orthogonality constraints were the most efficient in extracting sensory information, whereas non-negative tensor decompositions worked well even on non-independent and overlapping spike patterns, and retrieved informative firing patterns expressed by the same population in response to novel stimuli. Our method showed that populations of retinal ganglion cells carried information in their spike timing on the ten-milliseconds-scale about spatial details of natural images. This information could not be recovered from the spike counts of these cells. First-spike latencies carried the majority of information provided by the whole spike train about fine-scale image features, and supplied almost as much information about coarse natural image features as firing rates. Together, these results highlight the importance of spike timing, and particularly of first-spike latencies, in retinal coding.

Space-by-time manifold representation of dynamic facial expressions for emotion categorization.

Visual categorization is the brain computation that reduces high-dimensional information in the visual environment into a smaller set of meaningful categories. An important problem in visual neuroscience is to identify the visual information that the brain must represent and then use to categorize visual inputs. Here we introduce a new mathematical formalism-termed space-by-time manifold decomposition-that describes this information as a low-dimensional manifold separable in space and time. We use this decomposition to characterize the representations used by observers to categorize the six classic facial expressions of emotion (happy, surprise, fear, disgust, anger, and sad). By means of a Generative Face Grammar, we presented random dynamic facial movements on each experimental trial and used subjective human perception to identify the facial movements that correlate with each emotion category. When the random movements projected onto the categorization manifold region corresponding to one of the emotion categories, observers categorized the stimulus accordingly; otherwise they selected "other." Using this information, we determined both the Action Unit and temporal components whose linear combinations lead to reliable categorization of each emotion. In a validation experiment, we confirmed the psychological validity of the resulting space-by-time manifold representation. Finally, we demonstrated the importance of temporal sequencing for accurate emotion categorization and identified the temporal dynamics of Action Unit components that cause typical confusions between specific emotions (e.g., fear and surprise) as well as those resolving these confusions.