How mind wandering influences motor control: The modulating role of movement difficulty.

It has been found that mind wandering can impair motor control. However, it remains unclear whether the impact of mind wandering on motor control is modulated by movement difficulty and its associated neural mechanisms. To address this issue, we manipulated movement difficulty using handedness and finger dexterity separately in two signal-response tasks with identical experiment designs, in which right-handed participants performed key-pressing and key-releasing movements with the specified fingers, and they had to intermittently report whether their attention was "On task" or "Off task." Key-releasing with the right index finger (RI) had a faster reaction time and stronger contralateral delta-theta (1-7 Hz) functional connectivity than with the left index (LI) in Experiment 1, and mind wandering only reduced the contralateral delta-theta functional connectivity and midfrontal delta-theta activity for key-releasing with RI. Key-pressing with right index and middle fingers (RIR) had a faster reaction time and stronger midfrontal delta-theta activity than with right index and ring fingers (RIR) in Experiment 2, and mind wandering only reduced the midfrontal delta-theta activity for key-pressing with RIM. Theta oscillations are vital in motor control. These findings suggest that mind wandering only impairs the motor control of relatively simple movements without affecting the difficult ones. It supports the notion that mind wandering competes for executive resources with the primary task. Moreover, the quantity of executive resources recruited for a task and how these resources are allocated is contingent upon the task difficulty, which may determine whether mind wandering would interfere with motor control.

The Visual Advantage Effect in Comparing Uni-Modal and Cross-Modal Probabilistic Category Learning.

People rely on multiple learning systems to complete weather prediction (WP) tasks with visual cues. However, how people perform in audio and audiovisual modalities remains elusive. The present research investigated how the cue modality influences performance in probabilistic category learning and conscious awareness about the category knowledge acquired. A modified weather prediction task was adopted, in which the cues included two dimensions from visual, auditory, or audiovisual modalities. The results of all three experiments revealed better performances in the visual modality relative to the audio and audiovisual modalities. Moreover, participants primarily acquired unconscious knowledge in the audio and audiovisual modalities, while conscious knowledge was acquired in the visual modality. Interestingly, factors such as the amount of training, the complexity of visual stimuli, and the number of objects to which the two cues belonged influenced the amount of conscious knowledge acquired but did not change the visual advantage effect. These findings suggest that individuals can learn probabilistic cues and category associations across different modalities, but a robust visual advantage persists. Specifically, visual associations can be learned more effectively, and are more likely to become conscious. The possible causes and implications of these effects are discussed.

The roles of edge-based and surface-based information in the dynamic neural representation of objects.

We combined multivariate pattern analysis (MVPA) and electroencephalogram (EEG) to investigate the role of edge, color, and other surface information in the neural representation of visual objects. Participants completed a one-back task in which they were presented with color photographs, grayscale images, and line drawings of animals, tools, and fruits. Our results provide the first neural evidence that line drawings elicit similar neural activities as color photographs and grayscale images during the 175-305 ms window after the stimulus onset. Furthermore, we found that other surface information, rather than color information, facilitates decoding accuracy in the early stages of object representations and affects the speed of this. These results provide new insights into the role of edge-based and surface-based information in the dynamic process of neural representations of visual objects.

Word Familiarity Modulates the Interference Effects of Mind Wandering on Semantic and Reafferent Information Processing.

It has been found that mind wandering interferes with the sensory and cognitive processing of widespread stimuli. However, it remains unclear what factors can modulate the magnitude of the interference effects of mind wandering. Here, we investigate whether and how word familiarity modulates the interference effects of mind wandering on semantic and reafferent information processing. High- and low-frequency words were used as stimuli to induce high- and low-familiarity contexts in a sustained attention to response task, in which participants were required to respond to Chinese nonanimal words (nontarget) and withhold responses to Chinese animal words (target) as well as to intermittently report whether their state was "on task" or "off task." Behavioral results revealed lower reaction stability for both high- and low-frequency nontarget words preceding "off-task" reports than those preceding "on-task" reports. However, ERP results revealed that low-frequency rather than high-frequency words elicited more negative N400, attenuated late positive complex, and attenuated reafferent potential for "off-task" reports than for "on-task" reports. The results suggest that mind wandering makes semantic extraction and integration more difficult for unfamiliar but not familiar two-character Chinese words and attenuates the reafferent feedback of the motor response. These findings are consistent with the decoupling hypothesis of mind wandering and provide the first neural evidence for how familiarity with external stimuli modulates the interference effects of mind wandering.

How do irrelevant stimuli from another modality influence responses to the targets in a same-different task.

It remains unclear whether multisensory interaction can implicitly occur at the abstract level. To address this issue, a same-different task was used to select comparable images and sounds in Experiment 1. Then, the stimuli with various levels of discrimination difficulty were adopted in a modified same-different task in Experiments 2, 3, and 4. The resultsshowed that only when the irrelevant stimuli were easily distinguishable, aconsistency effectcould beobservedin the testing phase. Moreover, when easily distinguishableirrelevant stimuliwere simultaneously presented with difficulttarget stimuli, irrelevant auditorystimuli facilitated responses to visual targets whereas irrelevant visual stimuli interfered with responses to auditorytargetsin the training phase,indicating an asymmetry in the role of visual and auditory in abstract multisensory integration. The results suggested that abstract multisensory information could be implicitly integrated and the inverse effectiveness principle might not apply to high-level processing of abstract multisensory integration.

Multisensory transfer effects in implicit and explicit category learning.

In category learning, transfer emerges when individuals apply the knowledge or strategy learned with one set of stimuli to a novel set. Under multisensory circumstances, it remains unclear whether the transfer occurs differently in implicit and explicit category learning, as it remains controversial whether explicit knowledge is indispensable for transfer. To address this issue, we adopted multisensory stimuli in implicit (i.e., information integration, II) and explicit (i.e., conjunctive rule-based, RB) category learning tasks. Experiment 1 showed that when training and novel sets shared the same feature modalities, spontaneous transfer without feedback was perfect in the II condition but impaired in the RB condition. However, with feedback, the dependent transfer occurred equally between conditions. Experiment 2 revealed that when across different feature modalities, no spontaneous transfer was observed in any of the two conditions. A dependent transfer occurred in the RB but not in the II condition. Experiment 3 found that delayed feedback disrupted II task performance but not RB task performance, indicating that people did not learn the II task using an explicit rule. The current study demonstrated that transfer to novel stimuli could emerge in explicit and implicit category learning within feature modalities. However, the transfer across feature modalities could only arise with feedback in explicit learning. We further discuss these findings' meaning with the current category learning theory.

The role of working memory and visual processing in prototype category learning.

To investigate whetherworking memory and visual processing havethe same role or differentrolesin A/B and A/not A prototype category learning,the present study adoptedan A/Bor A/not A category learningtask in control and dual conditions. The results of Experiment 1 showed that an additional dual visual working memory taskrather thanadualverbal working memory task reduced accuracy of the A/B task, whereasnodual tasksinfluencedaccuracy of the A/not A task. The results of Experiment 2 revealed that an additionaldual visual processing task impairedaccuracy of the A/B task, whereas the dual visual processing task did not influence accuracy of the A/not Atask. These results indicate that visual working memory and visual processing play different roles in A/B and A/not A prototype category learning, andsupport that thesetwo types of prototype category learning are mediated by differentmemory systems.

Multisensory Information Facilitates the Categorization of Untrained Stimuli.

Although it has been demonstrated that multisensory information can facilitate object recognition and object memory, it remains unclear whether such facilitation effect exists in category learning. To address this issue, comparable car images and sounds were first selected by a discrimination task in Experiment 1. Then, those selected images and sounds were utilized in a prototype category learning task in Experiments 2 and 3, in which participants were trained with auditory, visual, and audiovisual stimuli, and were tested with trained or untrained stimuli within the same categories presented alone or accompanied with a congruent or incongruent stimulus in the other modality. In Experiment 2, when low-distortion stimuli (more similar to the prototypes) were trained, there was higher accuracy for audiovisual trials than visual trials, but no significant difference between audiovisual and auditory trials. During testing, accuracy was significantly higher for congruent trials than unisensory or incongruent trials, and the congruency effect was larger for untrained high-distortion stimuli than trained low-distortion stimuli. In Experiment 3, when high-distortion stimuli (less similar to the prototypes) were trained, there was higher accuracy for audiovisual trials than visual or auditory trials, and the congruency effect was larger for trained high-distortion stimuli than untrained low-distortion stimuli during testing. These findings demonstrated that higher degree of stimuli distortion resulted in more robust multisensory effect, and the categorization of not only trained but also untrained stimuli in one modality could be influenced by an accompanying stimulus in the other modality.

A Dual Simple Recurrent Network Model for Chunking and Abstract Processes in Sequence Learning.

Although many studies have provided evidence that abstract knowledge can be acquired in artificial grammar learning, it remains unclear how abstract knowledge can be attained in sequence learning. To address this issue, we proposed a dual simple recurrent network (DSRN) model that includes a surface SRN encoding and predicting the surface properties of stimuli and an abstract SRN encoding and predicting the abstract properties of stimuli. The results of Simulations 1 and 2 showed that the DSRN model can account for learning effects in the serial reaction time (SRT) task under different conditions, and the manipulation of the contribution weight of each SRN accounted for the contribution of conscious and unconscious processes in inclusion and exclusion tests in previous studies. The results of human performance in Simulation 3 provided further evidence that people can implicitly learn both chunking and abstract knowledge in sequence learning, and the results of Simulation 3 confirmed that the DSRN model can account for how people implicitly acquire the two types of knowledge in sequence learning. These findings extend the learning ability of the SRN model and help understand how different types of knowledge can be acquired implicitly in sequence learning.

The Role of Edge-Based and Surface-Based Information in Incidental Category Learning: Evidence From Behavior and Event-Related Potentials.

Although it has been demonstrated that edge-based information is more important than surface-based information in incidental category learning, it remains unclear how the two types of information play different roles in incidental category learning. To address this issue, the present study combined behavioral and event-related potential (ERP) techniques in an incidental category learning task in which the categories were defined by either edge- or surface-based features. The results from Experiment 1 showed that participants could simultaneously learn both edge- and surface-based information in incidental category learning, and importantly, there was a larger learning effect for the edge-based category than for the surface-based category. The behavioral results from Experiment 2 replicated those from Experiment 1, and the ERP results further revealed that the stimuli from the edge-based category elicited larger anterior and posterior P2 components than those from the surface-based category, whereas the stimuli from the surface-based category elicited larger anterior N1 and P3 components than those from the edge-based category. Taken together, the results suggest that, although surface-based information might attract more attention during feature detection, edge-based information plays more important roles in evaluating the relevance of information in making a decision in categorization.

Stimulus modality influences the acquisition and use of the rule-based strategy and the similarity-based strategy in category learning.

This study aimed to investigate whether stimulus modality influenced the acquisition and use of the rule-based strategy and the similarity-based strategy in category learning and whether the use of the two strategies was supported by shared or separate neural substrates. To address these issues, we combined behavioral and fNIRS methods in a modified prototype distortion task in which each category member has one rule feature and ten similarity features, and each type of feature can be presented in either the visual modality or the auditory modality. The results in Experiment 1 revealed that the learning effect in the "auditory rule-visual similarity" condition was the highest among all four conditions; further analysis revealed that in the "auditory rule-visual similarity" condition, the number of participants who used the rule-based strategy was more than the number of participants who used the similarity-based strategy, and the learning effect was always much higher for the rule-based strategy than for the similarity-based strategy. The behavioral results in Experiment 2 replicated the main findings in Experiment 1, and the fNIRS results showed that the use of the visual rule-based strategy was mediated by the dorsolateral prefrontal cortex, whereas the use of the auditory similarity-based strategy mainly engaged in the superior temporal gyrus, and the use of the visual similarity-based strategy mainly engaged in the inferior temporal gyrus. The results in Experiment 3 revealed that when the stimuli had only one type of feature, the visual rule rather than the auditory rule was learned more easily. The results provide new evidence that the stimulus modality can influence the acquisition and use of the rule-based strategy and the similarity-based strategy in category learning and that the use of the two types of strategies is supported by separate neural substrates both in the auditory modality and the visual modality.

Which Matters More in Incidental Category Learning: Edge-Based Versus Surface-Based Features.

Although many researches have shown that edge-based information is more important than surface-based information in object recognition, it remains unclear whether edge-based features play a more crucial role than surface-based features in category learning. To address this issue, a modified prototype distortion task was adopted in the present study, in which each category was defined by a rule or a similarity about either the edge-based features (i.e., contours or shapes) or the corresponding surface-based features (i.e., color and textures). The results of Experiments 1 and 2 showed that when the category was defined by a rule, the performance was significantly better in the edge-based condition than in the surface-based condition in the testing phase, and increasing the defined dimensions enhanced rather than reduced performance in the edge-based condition but not in the surface-based condition. The results of Experiment 3 showed that when each category was defined by a similarity, there was also a larger learning effect when the category was defined by edge-based dimensions than by surface-based dimensions in the testing phase. The current study is the first to provide convergent evidence that the edge-based information matters more than surface-based information in incidental category learning.

Dataset of implicit sequence learning of chunking and abstract structures.

This article describes the data analyzed in the paper "Implicit sequence learning of chunking and abstract structures" (Fu et al., 2018) [1]. It includes reaction times in the serial reaction time task and generation proformance for each confidence rating or attribution under the inclusion and exclusion tests from three experiments. For the serial reaction time task, the independent varialbles were type of stimuli and blocks or type of deviants; for the generation tests, the independent varialbles were type of stimuli, instructions, and confidence ratings or attribution tests. The data can be used to examine wether a computor model can account for what type of knowledge is acquried in implicit sequence learning.

Implicit acoustic sequence learning recruits the hippocampus.

The exclusive role of the medial temporal lobe in explicit memory has been questioned by several studies reporting medial temporal lobe involvement during implicit learning. Prior studies have demonstrated that hippocampal engagement is present during the implicit learning of perceptual associations, however, it is absent during learning response-related associations. Therefore, it was hypothesized that the function of the medial temporal lobe during implicit learning is related to the extraction of perceptual associations in general. While in most implicit learning tasks visual stimuli were used, the aim of the current functional magnetic resonance imaging (fMRI) study was to detect whether activations within medial temporal lobe structures are also found during implicit learning of auditory associations. In a modified version of the classical serial reaction time task, participants reacted to the presentation of five different tones. Unbeknownst to the participants, the tones were presented with an underlying sequential regularity that could be learned. To avoid an influence of response learning on acoustic associative learning, response buttons were remapped in every trial. After learning, two different tests were used to measure participants' conscious knowledge about the underlying sequence in order to assess the amount of implicit memory and to exclude participants with explicit knowledge acquired during learning. fMRI results revealed hippocampal activations for implicit learning of the acoustic sequence. When detecting a relation between implicit learning of acoustic associations and hippocampal activations, this study indicated a relation between hippocampal activations and memory formation of perceptual-based relational representation regardless of explicit knowledge. Thus, present findings suggest a general functional role for the formation of sequenced perceptual associations independent of the involvement of awareness.

Implicit sequence learning of chunking and abstract structures.

The current study investigated whether people can simultaneously acquire knowledge about concrete chunks and abstract structures in implicit sequence learning; and whether the degree of abstraction determines the conscious status of the acquired knowledge. We adopted three types of stimuli in a serial reaction time task in three experiments. The RT results indicated that people could simultaneously acquire knowledge about concrete chunks and abstract structures of the temporal sequence. Generation performance revealed that ability to control was mainly based on abstract structures rather than concrete chunks. Moreover, ability to control was not generally accompanied with awareness of knowing or knowledge, as measured by confidence ratings and attribution tests, confirming that people could control the use of unconscious knowledge of abstract structures. The results present a challenge to computational models and theories of implicit learning.

Emotional context modulates micro-expression processing as reflected in event-related potentials.

Micro-expression recognition is influenced by emotional contexts at the behavioral level. It is found that the recognition for micro-expressions is poorer following a negative context compared to the recognition following a neutral context. However, it remains unclear whether the modulation takes place in the early or later stage of the processing of micro-expressions as reflected in event-related potentials (ERPs). Using synthesized micro-expressions in this study, we investigated how emotional context modulates the ERP components that are elicited by micro-expressions. We observed that, compared to neutral contexts, negative and positive contexts elicited more positive amplitude trends for processing the subsequent target micro-expressions, and more positive P1 and N170 effects were elicited by target micro-expressions following negative and positive contexts compared to those following neutral contexts. Moreover, there were larger P1 effects elicited by the target micro-expressions when the context and target expressions were incongruent compared to when they were congruent. Our findings provide new ERP evidence to indicate that emotional contexts modulate micro-expression processing, and that the modulation of emotional contexts takes place at the early stage of facial-expression processing.

Neural Correlates of Subjective Awareness for Natural Scene Categorization of Color Photographs and Line-Drawings.

It remains controversial whether visual awareness is correlated with early activation indicated by VAN (visual awareness negativity), as the recurrent process hypothesis theory proposes, or with later activation indicated by P3 or LP (late positive), as suggested by global workspace theories. To address this issue, a backward masking task was adopted, in which participants were first asked to categorize natural scenes of color photographs and line-drawings and then to rate the clarity of their visual experience on a Perceptual Awareness Scale (PAS). The interstimulus interval between the scene and the mask was manipulated. The behavioral results showed that categorization accuracy increased with PAS ratings for both color photographs and line-drawings, with no difference in accuracy between the two types of images for each rating, indicating that the experience rating reflected visibility. Importantly, the event-related potential (ERP) results revealed that for correct trials, the early posterior N1 and anterior P2 components changed with the PAS ratings for color photographs, but did not vary with the PAS ratings for line-drawings, indicating that the N1 and P2 do not always correlate with subjective visual awareness. Moreover, for both types of images, the anterior N2 and posterior VAN changed with the PAS ratings in a linear way, while the LP changed with the PAS ratings in a non-linear way, suggesting that these components relate to different types of subjective awareness. The results reconcile the apparently contradictory predictions of different theories and help to resolve the current debate on neural correlates of visual awareness.

Paired-Associate and Feedback-Based Weather Prediction Tasks Support Multiple Category Learning Systems.

It remains unclear whether probabilistic category learning in the feedback-based weather prediction task (FB-WPT) can be mediated by a non-declarative or procedural learning system. To address this issue, we compared the effects of training time and verbal working memory, which influence the declarative learning system but not the non-declarative learning system, in the FB and paired-associate (PA) WPTs, as the PA task recruits a declarative learning system. The results of Experiment 1 showed that the optimal accuracy in the PA condition was significantly decreased when the training time was reduced from 7 to 3 s, but this did not occur in the FB condition, although shortened training time impaired the acquisition of explicit knowledge in both conditions. The results of Experiment 2 showed that the concurrent working memory task impaired the optimal accuracy and the acquisition of explicit knowledge in the PA condition but did not influence the optimal accuracy or the acquisition of self-insight knowledge in the FB condition. The apparent dissociation results between the FB and PA conditions suggested that a non-declarative or procedural learning system is involved in the FB-WPT and provided new evidence for the multiple-systems theory of human category learning.

The role of edge-based and surface-based information in natural scene categorization: Evidence from behavior and event-related potentials.

A fundamental question in vision research is whether visual recognition is determined by edge-based information (e.g., edge, line, and conjunction) or surface-based information (e.g., color, brightness, and texture). To investigate this question, we manipulated the stimulus onset asynchrony (SOA) between the scene and the mask in a backward masking task of natural scene categorization. The behavioral results showed that correct classification was higher for line-drawings than for color photographs when the SOA was 13ms, but lower when the SOA was longer. The ERP results revealed that most latencies of early components were shorter for the line-drawings than for the color photographs, and the latencies gradually increased with the SOA for the color photographs but not for the line-drawings. The results provide new evidence that edge-based information is the primary determinant of natural scene categorization, receiving priority processing; by contrast, surface information takes longer to facilitate natural scene categorization.

Emotional context influences micro-expression recognition.

Micro-expressions are often embedded in a flow of expressions including both neutral and other facial expressions. However, it remains unclear whether the types of facial expressions appearing before and after the micro-expression, i.e., the emotional context, influence micro-expression recognition. To address this question, the present study used a modified METT (Micro-Expression Training Tool) paradigm that required participants to recognize the target micro-expressions presented briefly between two identical emotional faces. The results of Experiments 1 and 2 showed that negative context impaired the recognition of micro-expressions regardless of the duration of the target micro-expression. Stimulus-difference between the context and target micro-expression was accounted for in Experiment 3. Results showed that a context effect on micro-expression recognition persists even when the stimulus similarity between the context and target micro-expressions was controlled. Therefore, our results not only provided evidence for the context effect on micro-expression recognition but also suggested that the context effect might result from both the stimulus and valence differences.