Prediction error and event segmentation in episodic memory.

Organizing the continuous flow of experiences into meaningful events is a crucial prerequisite for episodic memory. Prediction error and event segmentation both play important roles in supporting the genesis of meaningful mnemonic representations of events. We review theoretical contributions discussing the relationship between prediction error and event segmentation, as well as literature on episodic memory related to prediction error and event segmentation. We discuss the extent of overlap of mechanisms underlying memory emergence through prediction error and event segmentation, with a specific focus on attention and working memory. Finally, we identify areas in research that are currently developing and suggest future directions. We provide an overview of mechanisms underlying memory formation through predictions, violations of predictions, and event segmentation.

Not what u expect: Effects of prediction errors on item memory.

The characterization of the relationship between predictions and one-shot episodic encoding poses an important challenge for memory research. On the one hand, events that are compatible with our previous knowledge are thought to be remembered better than incompatible ones. On the other hand, unexpected situations, by virtue of their novelty, are known to cause enhanced learning. Several theoretical accounts try to solve this apparent paradox by conceptualizing prediction error (PE) as a continuum ranging from low PE (for expectation-matching events) to high PE (for expectation-mismatching ones). Under such a framework, the relationship between PE and memory encoding would be described by a U-shape function with higher memory performance for extreme levels of PE and lower memory for middle levels of PE. In this study, we tested the framework by using a gradual manipulation of the strength of association between scenes and objects to render different levels of PE and then tested for item memory of the (mis)matching events. In two experiments, in contrast to what was anticipated, recognition memory for object identity followed an inverted U-shape as a function of PE, with higher performance for intermediate levels of PE. Furthermore, in two additional experiments, we showed the relevance of explicit predictions at encoding to reveal such an inverted U pattern, thus providing the boundary conditions of the effect. We discussed our findings in light of existing literature relating PE and episodic memory, pointing out the potential roles of uncertainty in the environment, and the importance of the cognitive operations underlying encoding tasks. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Preparing auditory task switching in a task with overlapping and non-overlapping response sets.

We used a variant of cued auditory task switching to investigate task preparation and its relation to response-set overlap. Previous studies found increased interference with overlapping response sets across tasks relative to non-overlapping motor response sets. In the present experiments, participants classified either pitch or loudness of a simple tone as low or high, hence, both tasks were constructed around common underlying integrated semantic categories ranging from low to high. Manual responses overlapped in both category and modality for both tasks in Experiment 1A, whereas each task was related to a specific response category and response modality (manual vs. vocal) in Experiment 1B. Focusing on the manual responses in both experiments, the data showed that non-overlapping response sets (Experiment 1B) resulted in a decreased congruency effect, suggesting reduced response-based crosstalk and thus better task shielding, but at the same time switch costs were increased, suggesting less efficient switching between task sets. Moreover, varying preparation time (cue-stimulus interval, CSI) showed that long CSI led to better performance overall. Our results thus suggest that when non-overlapping response sets share common semantic categories across tasks, there is no general benefit over overlapping response sets.

The role of response set overlap for flexibility and cognitive control in auditory multitasking.

We developed a new variant of auditory task-switching in order to systematically investigate shifting and cognitive control in auditory task-switching and their relation to motor response overlap in a comprehensive way. In two experiments, participants classified either pitch or loudness of a simple tone as either low or high, hence, both tasks were constructed around a common underlying dimension ranging from low to high. In Experiment 1, response sets overlapped in both category and motor modality (both manual), whereas each task was related to a specific response category and motor response modality (manual vs. vocal) in Experiment 2. The data revealed reliable switch costs that were, contrary to our expectations, not reduced with reduced response set overlap. In addition, we found reliable congruency effects and their sequential modulation in both experiments with manual as well as vocal responses, and in the absence of competing motor activation (i.e., without motor response overlap). Congruency effects after auditory task switches were smaller when response sets did not overlap. Our data thus provides an important empirical generalization of known effects to auditory stimuli as well as with both manual and vocal responses. In addition, we demonstrated that reduced congruency effects after switches for non-overlapping response sets were due to the extent of overlap between different response sets in task-switching.

Children's intentional switching of auditory selective attention in spatial and noisy acoustic environments in comparison to adults.

Children's development and education take place in educational buildings with highly complex acoustic scenes, including spatially distributed target speakers, many surrounding distracting sounds, and general background noises. Auditory selective attention, therefore, is a valuable tool to orient oneself, to focus on specific sound sources, and to extract relevant information. Until now, it is unknown to what extent children have developed the cognitive processes of intentional attention control in spatial situations and how they differ from adults. This work provides a paradigm to examine children's intentional switching of auditory selective attention that also allows to examine effects due to noisy and spatial sound environments presented virtually via headphones. A listening experiment was conducted in Germany with 24 children (6-10 years, 50% female) and 24 young adults (18-26 years, 50% female). First, results revealed higher error rates and lower reaction times in conditions with noise (relative to conditions without noise) for children, but not for adults. This assumes that children are more sensitive to noise and conclude faster with noise trials, taking errors into account. Second, although auditory attention flexibility reflected in attention switch and relevant information selection was comparable between children and adults, it was found that adults benefited from spatial cues when selecting the relevant information. This was not observed to the same extent in children. These results suggest that children's cognitive processes are affected at significantly lower noise levels than adults and that noise effect assessment methods should consider spatial aspects. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Neural Correlates of Successful Memory Encoding in Kindergarten and Early Elementary School Children: Longitudinal Trends and Effects of Schooling.

From age 5 to 7, there are remarkable improvements in children's cognitive abilities ("5-7 shift"). In many countries, including Germany, formal schooling begins in this age range. It is, thus, unclear to what extent exposure to formal schooling contributes to the "5-7 shift." In this longitudinal study, we investigated if schooling acts as a catalyst of maturation. We tested 5-year-old children who were born close to the official cutoff date for school entry and who were still attending a play-oriented kindergarten. One year later, the children were tested again. Some of the children had experienced their first year of schooling whereas the others had remained in kindergarten. Using 2 functional magnetic resonance imaging tasks that assessed episodic memory formation (i.e., subsequent memory effect), we found that children relied strongly on the medial temporal lobe (MTL) at both time points but not on the prefrontal cortex (PFC). In contrast, older children and adults typically show subsequent memory effects in both MTL and PFC. Both children groups improved in their memory performance, but there were no longitudinal changes nor group differences in neural activation. We conclude that successful memory formation in this age group relies more heavily on the MTL than in older age groups.

The persisting influence of unattended auditory information: Negative priming in intentional auditory attention switching.

We studied negative priming (NP) in auditory attention switching. In a cued variant of dichotic listening, two spoken number words were presented, one to each ear, one spoken by a female, and one spoken by a male voice. A visual cue indicated whether the male or female voice was the target. A numerical magnitude judgement of the target number was required. The selection criterion could either switch or repeat across trials, so there were attention switch and repetition trials. Two experiments examined NP (distractor becomes target) and also included a "competitor priming" (CP) condition (target becomes distractor), relative to a "no priming" condition (target and distractor not related to previous trial). In Experiment 1, we investigated the basic priming effects. In Experiment 2, we additionally varied the response-cue interval (RCI; 100 ms vs. 1,900 ms) to examine time-related changes in priming. We found longer response times (RT) for switch trials compared with repetition trials (attention switch costs)-that is, when the internal processing context changed. In addition, we found longer RT for NP trials as well as reduced switch costs in long RCI, suggesting that previously relevant attentional settings dissipate over longer time. However, NP was not influenced by attention switches, and it was also not affected by RCI. Hence, NP in auditory attention switching does not seem strongly context or time sensitive.

Response repetitions in auditory task switching: The influence of spatial response distance and of the response-stimulus interval.

In task switching studies, response repetition effects are typically obtained: When the task repeats, response repetitions are faster than response switches (response repetition benefit), but when the task switches, the opposite is found (response repetition cost). Previously, it was found that spatial response distance [RD] affected the response repetitions: separated response keys led to longer reaction times [RT] for response repetitions (in both task repetitions and task switches) than adjacent response keys. The goal of the present study was to replicate this RD effect in a modified setup with auditory stimuli (in Experiments 1 and 2). As we were interested in the temporal dynamics of the RD effect, we also introduced a block-wise manipulation of response-stimulus interval (RSI) in Experiment 2. RD modulated responding, replicating the results of a prior study that used visual stimuli, but only when the RSI was long. With short RSI, the RD effect was not obtained. At the same time, a long RSI led to more pronounced response repetition effects in the error rates. These results imply that response inhibition from the previous trial, which is assumed to contribute to the response repetition effect and to the modulation of responding by response distance, builds up over time.

The binding of an auditory target location to a judgement: A two-component switching approach.

In a two-component switching paradigm, in which participants switched between two auditory attention selection criteria (attention component: left vs. right ear) and two judgements (judgement component: number vs. letter judgement), we found high judgement switch costs in attention criterion repetitions, but low costs in attention criterion switches. This finding showed an interaction of components. Previous two-component switching studies observed differently emphasised interaction patterns. In the present study, we explored whether the strength of the interaction pattern reflects the strength of the binding of target location and judgement. Specifically, we investigated whether exogenous target location cueing led to weaker binding than endogenous cueing, and whether preparation for ear selection could influence the binding. Attention switches with auditory exogenous target location cues did not affect the component interaction pattern, whereas a prolonged preparation interval led to a more emphasised pattern. Binding between target location and judgement may therefore be rather automatic and may not necessarily require concurrent component processing. Sufficient time for target location switches with long preparation time may activate the previous trial's episode or facilitate switches of the subsequent judgement.

Cognitive control in the cocktail party: Preparing selective attention to dichotically presented voices supports distractor suppression.

The goal of the present study was to investigate preparatory mechanisms of auditory selective attention. In two experiments, participants performed a classification task on one of two dichotically presented spoken number words, one spoken by a female, one spoken by a male. A cue indicated which gender participants had to attend to in the upcoming trial, so that attention switches and repetitions occurred randomly. The cue-target interval (CTI) was either 400 ms or 1,200 ms. Stimulus onset asynchrony (SOA) between target and distractor word varied; hence, the distractor could be presented before or after the target. In two experiments, we found robust performance costs of attention switches. Like in previous studies using versions of this paradigm, these switch costs were not significantly reduced by prolonged CTI, even though we found substantial general cue-based preparation effects. The most important finding refers to the influence of SOA, showing that the general preparation effect was greater in the condition with the distractor presented first than in the condition with the target presented first. Thus, increased time to prepare for the attention focus of the upcoming trials seems to benefit distractor suppression more than target enhancement. This occurred in switch trials and repetition trials alike, suggesting that it is a general feature of auditory attention.

Auditory attention switching and judgment switching: Exploring multicomponent task representations.

An auditory attention-switching paradigm was combined with a judgment-switching paradigm to examine the interaction of a varying auditory attention component and a varying judgment component. Participants heard two dichotically presented stimuli-one spoken by a female speaker and one spoken by a male speaker. In each trial, the stimuli were a spoken letter and a spoken number. A visual explicit cue at the beginning of each trial indicated the auditory attention criterion (speaker sex/ear) to identify the target stimulus (Experiment 1) or the judgment that had to be executed (Experiment 2). Hence, the attentional selection criterion switched independently between speaker sexes (or between ears), while the judgment alternated between letter categorization and number categorization. The data indicate that auditory attention criterion and judgment were not processed independently, regardless of whether the attention criterion or the judgment was cued. The partial repetition benefits of the explicitly cued component suggested a hierarchical organization of the auditory attention component and the judgment component within the task set. We suggest that the hierarchy arises due to the explicit cuing of one component rather than due to a "natural" hierarchy of auditory attention component and judgment component.

Common Cognitive Control Processes Underlying Performance in Task-Switching and Dual-Task Contexts.

In the present study, participants performed highly comparable task-switching and dual-task paradigms, and the paradigm-specific performance costs were analysed in the context of the commonly postulated core components of cognitive control (i.e., working memory updating, inhibition, and shifting). In the task-switching paradigm, we found switch costs (i.e., switch trials vs. repetition trials) and mixing costs (i.e., repetition trials in mixed-task blocks vs. single-task trials). In the dual-task paradigm, we observed a psychological refractory period (PRP) effect (i.e., Task 2 [T2] performance after short stimulus-onset asynchrony [SOA] vs. long SOA), dual-task costs (i.e., T2 dual-task performance with a long SOA in trials with a task repetition between Task 1 [T1] and T2 vs. single-task performance), and switch costs in T2 (i.e., dual-task performance in trials with a switch between T1 and T2 vs. dual-task performance in trials with a repetition between T1 and T2). A within-subjects comparison of the performance costs showed a correlation between mixing costs and dual-task costs, possibly indicating shared underlying cognitive control processes in terms of working memory updating. Surprisingly, there was also a correlation between switch costs and the PRP effect, presumably suggesting that cognitive control, as opposed to passive queuing of response selection processes, contributes to the PRP effect.

Intentional preparation of auditory attention-switches: Explicit cueing and sequential switch-predictability.

In an auditory attention-switching paradigm, participants heard two simultaneously spoken number-words, each presented to one ear, and decided whether the target number was smaller or larger than 5 by pressing a left or right key. An instructional cue in each trial indicated which feature had to be used to identify the target number (e.g., female voice). Auditory attention-switch costs were found when this feature changed compared to when it repeated in two consecutive trials. Earlier studies employing this paradigm showed mixed results when they examined whether such cued auditory attention-switches can be prepared actively during the cue-stimulus interval. This study systematically assessed which preconditions are necessary for the advance preparation of auditory attention-switches. Three experiments were conducted that controlled for cue-repetition benefits, modality switches between cue and stimuli, as well as for predictability of the switch-sequence. Only in the third experiment, in which predictability for an attention-switch was maximal due to a pre-instructed switch-sequence and predictable stimulus onsets, active switch-specific preparation was found. These results suggest that the cognitive system can prepare auditory attention-switches, and this preparation seems to be triggered primarily by the memorised switching-sequence and valid expectations about the time of target onset.

Hierarchical task organization in dual tasks: evidence for higher level task representations.

To examine whether hierarchical higher level task representations comprising the task sets of Task 1 (T1) and Task 2 (T2) are activated within each trial in dual-task situations, we combined the psychological refractory period (PRP) paradigm with the task-pair switching logic (Hirsch et al. 2017). In Experiment 1, in which subjects switched between task-pairs including a varying T1 and a constant T2, we found a PRP effect (i.e., worse performance with short stimulus onset asynchrony [SOA] than with long SOA) and task-pair switch costs in T1 and T2 (impaired performance in task-pair switches compared to task-pair repetitions). However, since in Experiment 1 there were no forward and backward response-response compatibility effects that indicated interference between T1 and T2, we could not exclude that the activation of T1 persisted into the next trial despite the intervening T2, and hence, that task-pair switch costs are due to repetition-priming effects of T1 across task-pairs rather than due to persisting activation of task-pair representations. In Experiment 2, we used a modified task-pair switching logic with a constant T1 and a varying T2, and replicated task-pair switch costs under conditions that not only rule out repetition-priming effects of T1 across task-pairs as the source of task-pair switch costs but also disentangle the effects of switching task-pairs from those of switching T1. These effects were associated in previous studies using the original task-pair switching logic. Thus, the findings of the present study strongly suggest that hierarchical higher level task representations are activated during dual-task processing.

Effects of musical expertise on oscillatory brain activity in response to emotional sounds.

Emotions can be conveyed through a variety of channels in the auditory domain, be it via music, non-linguistic vocalizations, or speech prosody. Moreover, recent studies suggest that expertise in one sound category can impact the processing of emotional sounds in other sound categories as they found that musicians process more efficiently emotional musical and vocal sounds than non-musicians. However, the neural correlates of these modulations, especially their time course, are not very well understood. Consequently, we focused here on how the neural processing of emotional information varies as a function of sound category and expertise of participants. Electroencephalogram (EEG) of 20 non-musicians and 17 musicians was recorded while they listened to vocal (speech and vocalizations) and musical sounds. The amplitude of EEG-oscillatory activity in the theta, alpha, beta, and gamma band was quantified and Independent Component Analysis (ICA) was used to identify underlying components of brain activity in each band. Category differences were found in theta and alpha bands, due to larger responses to music and speech than to vocalizations, and in posterior beta, mainly due to differential processing of speech. In addition, we observed greater activation in frontal theta and alpha for musicians than for non-musicians, as well as an interaction between expertise and emotional content of sounds in frontal alpha. The results reflect musicians' expertise in recognition of emotion-conveying music, which seems to also generalize to emotional expressions conveyed by the human voice, in line with previous accounts of effects of expertise on musical and vocal sounds processing.

Intentional switching of auditory attention between long and short sequential tone patterns.

The current study focuses on auditory task switching, more precisely on switching attention between different temporal patterns of the same auditory stimulus. Tone sequences consisting of nine different pitch tones were presented aurally. Three repetitive short 3-tone patterns (local focus) were combined to a long pattern (global focus), and each could be either rising or falling, resulting in congruent or incongruent combinations. Participants were informed by a cue if they had to attend to the short or to the long pattern, and they indicated if the target pattern was rising or falling by pressing one of two keys. In two experiments, we investigated cued switches between the two attentional foci. Switch costs in reaction times and errors were observed when switching from the long to the short pattern but not when switching from the short to the long pattern. These asymmetric switch costs were reduced when participants had more time to prepare for the switch in a condition with a prolonged cue-stimulus interval. In addition, participants made more errors when global and local patterns did not correspond to each other (i.e., in incongruent trials) when attending to either of the patterns, but this congruency effect was not modulated by preparation time. The data suggest that the mechanisms of task goal prioritizing, as indicated by the asymmetric attention switch costs, are dissociable from those underlying stimulus selection, as indicated by the congruency effects.

Higher-order cognitive control in dual tasks: Evidence from task-pair switching.

In the present study, we combined the psychological refractory period (PRP) paradigm with a novel task-pair switching logic which enabled us to isolate performance costs occurring at the global level of task-pairs. In Experiment 1, in which we used conceptually overlapping responses for Task 1 (T1) and Task 2 (T2), we generated 3 task-pairs by combining 1 of 3 visual tasks (T1) with an auditory task (T2). In addition to worse performance after a short SOA than a long SOA (i.e., PRP effect), we found impaired performance in n - 1 task-pair switches as compared to n - 1 task-pair repetitions (i.e., n - 1 task-pair switch costs), suggesting that task-pairs were activated during dual-task processing. In Experiment 2, we increased the interference between T1 and T2 by using physically overlapping responses and we again observed n - 1 task-pair switch costs. To investigate whether the activation of task-pairs is adjusted by inhibitory control, we looked at the n - 2 task-pair sequence and found performance to be better in n - 2 task-pair repetitions than in n - 2 task-pair switches in both experiments. This n - 2 task-pair repetition benefit was replicated in Experiment 3 in which no immediate task-pair repetitions were included. Hence, the evidence suggests enhanced activation rather than inhibition as a crucial selection mechanism at the global level of dual-task processing. (PsycINFO Database Record

No differences in dual-task costs between forced- and free-choice tasks.

Humans appear to act in response to environmental demands or to pursue self-chosen goals. In the laboratory, these situations are often investigated with forced- and free-choice tasks: in forced-choice tasks, a stimulus determines the one correct response, while in free-choice tasks the participants choose between response alternatives. We compared these two tasks regarding their susceptibility to dual-task interference when the concurrent task was always forced-choice. If, as was suggested in the literature, both tasks require different "action control systems," larger dual-task costs for free-choice tasks than for forced-choice tasks should emerge in our experiments, due to a time-costly switch between the systems. In addition, forced-choice tasks have been conceived as "prepared reflexes" for which all intentional processing is said to take place already prior to stimulus onset giving rise to automatic response initiation upon stimulus onset. We report three experiments with different implementations of the forced- vs. free-choice manipulation. In all experiments we replicated slower responses in the free- than in the forced-choice task and the typical dual-task costs. These latter costs, however, were equivalent for forced- and free-choice tasks. These results are easier to reconcile with the assumption of one unitary "action control system."

Saturation of auditory short-term memory causes a plateau in the sustained anterior negativity event-related potential.

The maintenance of information in auditory short-term memory (ASTM) is accompanied by a sustained anterior negativity (SAN) in the event-related potential measured during the retention interval of simple auditory memory tasks. Previous work on ASTM showed that the amplitude of the SAN increased in negativity as the number of maintained items increases. The aim of the current study was to measure the SAN and observe its behavior beyond the point of saturation of auditory short-term memory. We used atonal pure tones in sequences of 2, 4, 6, or 8t. Our results showed that the amplitude of SAN increased in negativity from 2 to 4 items and then levelled off from 4 to 8 items. Behavioral results suggested that the average span in the task was slightly below 3, which was consistent with the observed plateau in the electrophysiological results. Furthermore, the amplitude of the SAN predicted individual differences in auditory memory capacity. The results support the hypothesis that the SAN is an electrophysiological index of brain activity specifically related to the maintenance of auditory information in ASTM.

Brain activity is related to individual differences in the number of items stored in auditory short-term memory for pitch: evidence from magnetoencephalography.

We used magnetoencephalography (MEG) to examine brain activity related to the maintenance of non-verbal pitch information in auditory short-term memory (ASTM). We focused on brain activity that increased with the number of items effectively held in memory by the participants during the retention interval of an auditory memory task. We used very simple acoustic materials (i.e., pure tones that varied in pitch) that minimized activation from non-ASTM related systems. MEG revealed neural activity in frontal, temporal, and parietal cortices that increased with a greater number of items effectively held in memory by the participants during the maintenance of pitch representations in ASTM. The present results reinforce the functional role of frontal and temporal cortices in the retention of pitch information in ASTM. This is the first MEG study to provide both fine spatial localization and temporal resolution on the neural mechanisms of non-verbal ASTM for pitch in relation to individual differences in the capacity of ASTM. This research contributes to a comprehensive understanding of the mechanisms mediating the representation and maintenance of basic non-verbal auditory features in the human brain.