Consolidation Enhances Sequential Multistep Anticipation but Diminishes Access to Perceptual Features.

Many experiences unfold predictably over time. Memory for these temporal regularities enables anticipation of events multiple steps into the future. Because temporally predictable events repeat over days, weeks, and years, we must maintain-and potentially transform-memories of temporal structure to support adaptive behavior. We explored how individuals build durable models of temporal regularities to guide multistep anticipation. Healthy young adults (Experiment 1: N = 99, age range = 18-40 years; Experiment 2: N = 204, age range = 19-40 years) learned sequences of scene images that were predictable at the category level and contained incidental perceptual details. Individuals then anticipated upcoming scene categories multiple steps into the future, immediately and at a delay. Consolidation increased the efficiency of anticipation, particularly for events further in the future, but diminished access to perceptual features. Further, maintaining a link-based model of the sequence after consolidation improved anticipation accuracy. Consolidation may therefore promote efficient and durable models of temporal structure, thus facilitating anticipation of future events.

Using network science to provide insights into the structure of event knowledge.

The structure of event knowledge plays a critical role in prediction, reconstruction of memory for personal events, construction of possible future events, action, language usage, and social interactions. Despite numerous theoretical proposals such as scripts, schemas, and stories, the highly variable and rich nature of events and event knowledge have been formidable barriers to characterizing the structure of event knowledge in memory. We used network science to provide insights into the temporal structure of common events. Based on participants' production and ordering of the activities that make up events, we established empirical profiles for 80 common events to characterize the temporal structure of activities. We used the event networks to investigate multiple issues regarding the variability in the richness and complexity of people's knowledge of common events, including: the temporal structure of events; event prototypes that might emerge from learning across many experiential instances and be expressed by people; the degree to which scenes (communities) are present in various events; the degree to which people believe certain activities are central to an event; how centrality might be distributed across an event's activities; and similarities among events in terms of their content and their temporal structure. Thus, we provide novel insights into human event knowledge, and describe 18 predictions for future human studies.

A method to find temporal structure of neuronal coactivity patterns with across-trial correlations.

BACKGROUND: The across-trial correlation of neurons' coactivity patterns emerges to be important for information coding, but methods for finding their temporal structures remain largely unexplored. NEW METHOD: In the present study, we propose a method to find time clusters in which coactivity patterns of neurons are correlated across trials. We transform the multidimensional neural activity at each timing into a coactivity pattern of binary states, and predict the coactivity patterns at different timings. We devise a method suitable for these coactivity pattern predictions, call general event prediction. Cross-temporal prediction accuracy is then used to estimate across-trial correlations between coactivity patterns at two timings. We extract time clusters from the cross-temporal prediction accuracy by a modified k-means algorithm. RESULTS: The feasibility of the proposed method is verified through simulations based on ground truth. We apply the proposed method to a calcium imaging dataset recorded from the motor cortex of mice, and demonstrate time clusters of motor cortical coactivity patterns during a motor task. COMPARISON WITH EXISTING METHODS: While the existing cosine similarity method, which does not account for across-trial correlation, shows temporal structures only for contralateral neural responses, the proposed method reveals those for both contralateral and ipsilateral neural responses, demonstrating the effect of across-trial correlations. CONCLUSIONS: This study introduces a novel method for measuring the temporal structure of neuronal ensemble activity.

Temporal and spatial variations in the sub-daily precipitation structure over the Qinghai-Tibet Plateau (QTP).

Precipitation is a vital component of the global atmospheric and hydrological cycles and influencing the distribution of water resources. Even subtle changes in precipitation can significantly impact ecosystems, energy cycles, agricultural production, and food security. Therefore, understanding the changes in the precipitation structure under climate change is essential. The Qinghai-Tibet Plateau (QTP) is a region sensitive to global climate change and profoundly impacts the atmospheric water cycle in Asia and even globally, rendering it a hot topic in climate change research in recent years. Few studies have examined on the sub-daily scale precipitation structure over the QTP. In this paper, the characteristics of sub-daily precipitation on the QTP were systematically investigated from multiple perspectives, including the concentration index, skewness (the third standardized moment of a distribution), and kurtosis (the fourth standardized moment of a distribution). The results indicated that the frequency of moderate-intensity nighttime precipitation on the QTP generally increased, and the analysis of both the concentration index and kurtosis (skewness) suggested that extreme precipitation was more frequent in the southwestern foothills of the QTP. Furthermore, potential high-risk areas for natural disasters were identified on the QTP, and found that the southeastern part of the plateau constituted a potential hotspot area for flood disasters. Given the complexity of climate change, a comprehensive analysis of the spatiotemporal characteristics of diurnal and nighttime precipitation changes on the QTP could help reveal the regularity of precipitation changes. This has significant implications for forecasting, warning, disaster preparedness, and mitigation efforts on the QTP.

Temporal integration by multi-level regularities fosters the emergence of dynamic conscious experience.

The relationship between integration and awareness is central to contemporary theories and research on consciousness. Here, we investigated whether and how information integration over time, by incorporating the underlying regularities, contributes to our awareness of the dynamic world. Using binocular rivalry, we demonstrated that structured visual streams, constituted by shape, motion, or idiom sequences containing perceptual- or semantic-level regularities, predominated over their nonstructured but otherwise matched counterparts in the competition for visual awareness. Despite the apparent resemblance, a substantial dissociation of the observed rivalry advantages emerged between perceptual- and semantic-level regularities. These effects stem from nonconscious and conscious temporal integration processes, respectively, with the former but not the latter being vulnerable to perturbations in the spatiotemporal integration window. These findings corroborate the essential role of structure-guided information integration in visual awareness and highlight a multi-level mechanism where temporal integration by perceptually and semantically defined regularities fosters the emergence of continuous conscious experience.

Procedure for extracting temporal structure embedded within psychophysical data.

The idea that mental events unfold over time with an intrinsically paced regularity has a long history within experimental psychology, and it has gained traction from the actual measurement of brain rhythms evident in EEG signals recorded from the human brain and from direct recordings of action potentials and local field potentials within the nervous systems of nonhumans. The weak link in this idea, however, is the challenge of extracting signatures of this temporal structure from behavioral measures. Because there is nothing in the seamless stream of conscious awareness that belies rhythmic modulations in sensitivity or mental acuity, one must deploy inferential strategies for extracting evidence for the existence of temporal regularities in neural activity. We have devised a parametric procedure for analysis of temporal structure embedded in behaviorally measured data comprising durations. We confirm that this procedure, dubbed PATS, achieves comparable results to those obtained using spectral analysis, and that it outperforms conventional spectral analysis when analyzing human response time data containing just a few hundred data points per condition. PATS offers an efficient, sensitive means for bridging the gap between oscillations identified neurophysiologically and estimates of rhythmicity embedded within durations measured behaviorally.

Within arms reach: Physical proximity shapes mother-infant language exchanges in real-time.

During everyday interactions, mothers and infants achieve behavioral synchrony at multiple levels. The ebb-and-flow of mother-infant physical proximity may be a central type of synchrony that establishes a common ground for infant-mother interaction. However, the role of proximity in language exchanges is relatively unstudied, perhaps because structured tasks-the common setup for observing infant-caregiver interactions-establish proximity by design. We videorecorded 100 mothers (U.S. Hispanic N = 50, U.S. Non-Hispanic N = 50) and their 13- to 23-month-old infants during natural activity at home (1-to-2 h per dyad), transcribed mother and infant speech, and coded proximity continuously (i.e., infants and mother within arms reach). In both samples, dyads entered proximity in a bursty temporal pattern, with bouts of proximity interspersed with bouts of physical distance. As hypothesized, Non-Hispanic and Hispanic mothers produced more words and a greater variety of words when within arms reach than out of arms reach. Similarly, infants produced more utterances that contained words when close to mother than when not. However, infants babbled equally often regardless of proximity, generating abundant opportunities to play with sounds. Physical proximity expands opportunities for language exchanges and infants' communicative word use, although babies accumulate massive practice babbling even when caregivers are not proximal.

Efficient multi-task learning with adaptive temporal structure for progression prediction.

In this paper, we propose a novel efficient multi-task learning formulation for the class of progression problems in which its state will continuously change over time. To use the shared knowledge information between multiple tasks to improve performance, existing multi-task learning methods mainly focus on feature selection or optimizing the task relation structure. The feature selection methods usually fail to explore the complex relationship between tasks and thus have limited performance. The methods centring on optimizing the relation structure of tasks are not capable of selecting meaningful features and have a bi-convex objective function which results in high computation complexity of the associated optimization algorithm. Unlike these multi-task learning methods, motivated by a simple and direct idea that the state of a system at the current time point should be related to all previous time points, we first propose a novel relation structure, termed adaptive global temporal relation structure (AGTS). Then we integrate the widely used sparse group Lasso, fused Lasso with AGTS to propose a novel convex multi-task learning formulation that not only performs feature selection but also adaptively captures the global temporal task relatedness. Since the existence of three non-smooth penalties, the objective function is challenging to solve. We first design an optimization algorithm based on the alternating direction method of multipliers (ADMM). Considering that the worst-case convergence rate of ADMM is only sub-linear, we then devise an efficient algorithm based on the accelerated gradient method which has the optimal convergence rate among first-order methods. We show the proximal operator of several non-smooth penalties can be solved efficiently due to the special structure of our formulation. Experimental results on four real-world datasets demonstrate that our approach not only outperforms multiple baseline MTL methods in terms of effectiveness but also has high efficiency.

Neural mechanisms of temporal and rhythmic structure processing in non-musicians.

Music is increasingly being used as a therapeutic tool in the field of rehabilitation medicine and psychophysiology. One of the main key components of music is its temporal organization. The characteristics of neurocognitive processes during music perception of meter in different tempo variations technique have been studied by using the event-related potentials technique. The study involved 20 volunteers (6 men, the median age of the participants was 23 years). The participants were asked to listen to 4 experimental series that differed in tempo (fast vs. slow) and meter (duple vs. triple). Each series consisted of 625 audio stimuli, 85% of which were organized with a standard metric structure (standard stimulus) while 15% included unexpected accents (deviant stimulus). The results revealed that the type of metric structure influences the detection of the change in stimuli. The analysis showed that the N200 wave occurred significantly faster for stimuli with duple meter and fast tempo and was the slowest for stimuli with triple meter and fast pace.

The temporal structure of parent talk to toddlers about objects.

Toddlers learn words in the context of speech from adult social partners. The present studies quantitatively describe the temporal context of parent speech to toddlers about objects in individual real-world interactions. We show that at the temporal scale of a single play episode, parent talk to toddlers about individual objects is predominantly, but not always, clustered. Clustered speech is characterized by repeated references to the same object close in time, interspersed with lulls in speech about the object. Clustered temporal speech patterns mirror temporal patterns observed at longer timescales, and persisted regardless of play context. Moreover, clustered speech about individual novel objects predicted toddlers' learning of those objects' novel names. Clustered talk may be optimal for toddlers' word learning because it exploits domain-general principles of human memory and attention, principles that may have evolved precisely because of the clustered structure of natural events important to humans, including human behavior.

Revealing human sensitivity to a latent temporal structure of changes.

Precisely timed behavior and accurate time perception plays a critical role in our everyday lives, as our wellbeing and even survival can depend on well-timed decisions. Although the temporal structure of the world around us is essential for human decision making, we know surprisingly little about how representation of temporal structure of our everyday environment impacts decision making. How does the representation of temporal structure affect our ability to generate well-timed decisions? Here we address this question by using a well-established dynamic probabilistic learning task. Using computational modeling, we found that human subjects' beliefs about temporal structure are reflected in their choices to either exploit their current knowledge or to explore novel options. The model-based analysis illustrates a large within-group and within-subject heterogeneity. To explain these results, we propose a normative model for how temporal structure is used in decision making, based on the semi-Markov formalism in the active inference framework. We discuss potential key applications of the presented approach to the fields of cognitive phenotyping and computational psychiatry.

Knowledge of the temporal structure of events in relation to autistic traits and social ability.

Event knowledge, a person's understanding of patterns of activities in the world, is crucial for everyday social interactions. Social communication differences are prominent in autism, which may be related to atypical event knowledge, such as atypical knowledge of the sequences of activities that comprise the temporal structure of events. Previous research has found that autistic individuals have atypical event knowledge, but research in this area is minimal, particularly regarding autistic individuals' knowledge of the temporal structure of events. Furthermore, no studies have investigated the link between event knowledge and autistic traits in a non-clinical sample. We investigated relationships between event knowledge and autistic traits in individuals from the general population with varying degrees of autistic traits. We predicted that atypical ordering of event activities is related to autistic traits, particularly social communication abilities, but not other clinical traits. In Study 1, atypical ordering of event activities correlated with social ability, but not with most measures of repetitive behaviours and restricted interests. In Study 2, the typicality of activity ordering varied by participants' social ability and the social nature of the events. Relationships were not found between event activity ordering and other clinical traits. These findings suggest a relationship between autistic traits, specifically social abilities, and knowledge of the temporal structure of events in a general population sample.

Child language and autism diagnosis impact hierarchical temporal structure of parent-child vocal interactions in early childhood.

Timing is critical to successful social interactions. The temporal structure of dyadic vocal interactions emerges from the rhythm, timing, and frequency of each individuals' vocalizations and reflects how the dyad dynamically organizes and adapts during an interaction. This study investigated the temporal structure of vocal interactions longitudinally in parent-child dyads of typically developing (TD) infants (n = 49; 9-18 months; 48% male) and toddlers with ASD (n = 23; 27.2 ± 5.0 months; 91.3% male) to identify how developing language and social skills impact the temporal dynamics of the interaction. Acoustic hierarchical temporal structure (HTS), a measure of the nested clustering of acoustic events across multiple timescales, was measured in free play interactions using Allan Factor. HTS reflects a signal's temporal complexity and variability, with greater HTS indicating reduced flexibility of the dyadic system. Child expressive language significantly predicted HTS (ß = -0.2) longitudinally across TD infants, with greater dyadic HTS associated with lower child language skills. ASD dyads exhibited greater HTS (i.e., more rigid temporal structure) than nonverbal matched (d = 0.41) and expressive language matched TD dyads (d = 0.28). Increased HTS in ASD dyads occurred at timescales >1 s, suggesting greater structuring of pragmatic aspects of interaction. Results provide a new window into how language development and social reciprocity serve as constraints to shape parent-child interaction dynamics and showcase a novel automated approach to characterizing vocal interactions across multiple timescales during early childhood.

Geometrical representation of serial order in working memory.

Encoding a sequence relies on one's memory for ordinal succession of events and is critical for episodic memory, spatial navigation, language, and other cognitive functions. Investigating the neural mechanisms underlying sequence working memory in the macaque prefrontal cortex, Xie et al. (Science, 375, 632-639, 2022) uncovered a novel integrated representation of temporal and spatial information in different subspaces of a high-dimensional neural state space, offering broad implications across comparative cognition and neuroscience.

Modeling the Sequential Pattern Variability of the Electromotor Command System of Pulse Electric Fish.

Mormyridae, a family of weakly electric fish, use electric pulses for communication and for extracting information from the environment (active electroreception). The electromotor system controls the timing of pulse generation. Ethological studies have described several sequences of pulse intervals (SPIs) related to distinct behaviors (e.g., mating or exploratory behaviors). Accelerations, scallops, rasps, and cessations are four different SPI patterns reported in these fish, each showing characteristic stereotyped temporal structures. This article presents a computational model of the electromotor command circuit that reproduces a whole set of SPI patterns while keeping the same internal network configuration. The topology of the model is based on a simplified representation of the network with four neuron clusters (nuclei). An initial configuration was built to reproduce nucleus characteristics and network topology as described by detailed morphological and electrophysiological studies. Then, a methodology based on a genetic algorithm (GA) was developed and applied to tune the model connectivity parameters to automatically reproduce a whole set of patterns recorded from freely-behaving Gnathonemus petersii specimens. Robustness analyses of input variability were performed to discard overfitting and assess validity. Results show that the set of SPI patterns is consistently reproduced reaching a dynamic balance between synaptic properties in the network. This model can be used as a tool to test novel hypotheses regarding temporal structure in electrogeneration. Beyond the electromotor model itself, the proposed methodology can be adapted to fit models of other biological networks that also exhibit sequential patterns.

Modeling the Repetition-Based Recovering of Acoustic and Visual Sources With Dendritic Neurons.

In natural auditory environments, acoustic signals originate from the temporal superimposition of different sound sources. The problem of inferring individual sources from ambiguous mixtures of sounds is known as blind source decomposition. Experiments on humans have demonstrated that the auditory system can identify sound sources as repeating patterns embedded in the acoustic input. Source repetition produces temporal regularities that can be detected and used for segregation. Specifically, listeners can identify sounds occurring more than once across different mixtures, but not sounds heard only in a single mixture. However, whether such a behavior can be computationally modeled has not yet been explored. Here, we propose a biologically inspired computational model to perform blind source separation on sequences of mixtures of acoustic stimuli. Our method relies on a somatodendritic neuron model trained with a Hebbian-like learning rule which was originally conceived to detect spatio-temporal patterns recurring in synaptic inputs. We show that the segregation capabilities of our model are reminiscent of the features of human performance in a variety of experimental settings involving synthesized sounds with naturalistic properties. Furthermore, we extend the study to investigate the properties of segregation on task settings not yet explored with human subjects, namely natural sounds and images. Overall, our work suggests that somatodendritic neuron models offer a promising neuro-inspired learning strategy to account for the characteristics of the brain segregation capabilities as well as to make predictions on yet untested experimental settings.

Semi-supervised echo state network with temporal-spatial graph regularization for dynamic soft sensor modeling of industrial processes.

Echo state network (ESN) has been successfully applied to industrial soft sensor field because of its strong nonlinear and dynamic modeling capability. Nevertheless, the traditional ESN is intrinsically a supervised learning technique, which only depends on labeled samples, but omits a large number of unlabeled samples. In order to eliminate this limitation, this work proposes a semi-supervised ESN method assisted by a temporal-spatial graph regularization (TSG-SSESN) for constructing soft sensor model with all the available samples. Firstly, the traditional supervised ESN is enhanced to construct the semi-supervised ESN (SSESN) model by integrating both unlabeled and labeled samples in the reservoir computing procedure. The SSESN computes the reservoir states under high sampling rate for better process dynamic information mining. Furthermore, the SSESN's output optimization objective is modified by applying the local adjacency graph of all training samples as a regularization term. Especially, in view of the dynamic data characteristic, a temporal-spatial graph is constructed by considering both the temporal relationship and the spatial distances. The applications to a debutanizer column process and a wastewater treatment plant demonstrate that the TSG-SSESN model can build much smoother model and has better generalization capability than the basic ESN models in terms of soft sensor prediction results.

Coupling of Mouse Olfactory Bulb Projection Neurons to Fluctuating Odor Pulses.

Odors are transported by turbulent air currents, creating complex temporal fluctuations in odor concentration that provide a potentially informative stimulus dimension. We have shown that mice are able to discriminate odor stimuli based on their temporal structure, indicating that information contained in the temporal structure of odor plumes can be extracted by the mouse olfactory system. Here, using in vivo extracellular and intracellular electrophysiological recordings, we show that mitral cells (MCs) and tufted cells (TCs) of the male C57BL/6 mouse olfactory bulb can encode the dominant temporal frequencies present in odor stimuli up to at least 20 Hz. A substantial population of cell-odor pairs showed significant coupling of their subthreshold membrane potential with the odor stimulus at both 2 Hz (29/70) and the suprasniff frequency 20 Hz (24/70). Furthermore, mitral/tufted cells (M/TCs) show differential coupling of their membrane potential to odor concentration fluctuations with tufted cells coupling more strongly for the 20 Hz stimulation. Frequency coupling was always observed to be invariant to odor identity, and M/TCs that coupled well to a mixture also coupled to at least one of the components of the mixture. Interestingly, pharmacological blocking of the inhibitory circuitry strongly modulated frequency coupling of cell-odor pairs at both 2 Hz (10/15) and 20 Hz (9/15). These results provide insight into how both cellular and circuit properties contribute to the encoding of temporal odor features in the mouse olfactory bulb.SIGNIFICANCE STATEMENT Odors in the natural environment have a strong temporal structure that can be extracted and used by mice in their behavior. Here, using in vivo extracellular and intracellular electrophysiological techniques, we show that the projection neurons in the olfactory bulb can encode and couple to the dominant frequency present in an odor stimulus. Furthermore, frequency coupling was observed to be differential between mitral and tufted cells and was odor invariant but strongly modulated by local inhibitory circuits. In summary, this study provides insight into how both cellular and circuit properties modulate encoding of odor temporal features in the mouse olfactory bulb.

A guideline for linking brain wave findings to the various aspects of discrete perception.

Brain waves, determined by electrical and magnetic brain recordings (e.g., EEG and MEG), and fluctuating behavioral responses, determined by response time or accuracy measures, are frequently taken to support discrete perception. For example, it has been proposed that humans experience only one conscious percept per brain wave (e.g., during one alpha cycle). However, the proposed link between brain waves and discrete perception is typically rather vague. More importantly, there are many models and aspects of discrete perception and it is often not apparent in what theoretical framework brain wave findings are interpreted and to what specific aspects of discrete perception they relate. Here, we review different approaches to discrete perception and highlight issues with particular interpretations. We then discuss how certain findings on brain waves may relate to certain aspects of discrete perception. The main purpose of this meta-contribution is to give a short overview of discrete models of perception and to illustrate the need to make explicit what aspects of discrete theories are addressed by what aspects of brain wave findings.

About time: Ageing influences neural markers of temporal predictability.

Timing abilities help organizing the temporal structure of events but are known to change systematically with age. Yet, how the neuronal signature of temporal predictability changes across the age span remains unclear. Younger (n = 21; 23.1 years) and older adults (n = 21; 68.5 years) performed an auditory oddball task, consisting of isochronous and random sound sequences. Results confirm an altered P50 response in the older compared to younger participants. P50 amplitudes differed between the isochronous and random temporal structures in younger, and for P200 in the older group. These results suggest less efficient sensory gating in older adults in both isochronous and random auditory sequences. N100 amplitudes were more negative for deviant tones. P300 amplitudes were parietally enhanced in younger, but not in older adults. In younger participants, the P50 results confirm that this component marks temporal predictability, indicating sensitive gating of temporally regular sound sequences.