Construction or updating? Event model processes during visual narrative comprehension.

The plot of a narrative is represented in the form of event models in working memory. Because only parts of the plot are actually presented and information is continually changing, comprehenders have to infer a good portion of a narrative and keep their mental representation updated. Research has identified two related processes (e.g., Gernsbacher, 1997): During model construction (shifting, laying a foundation) at large coherence breaks an event model is completely built anew. During model updating (mapping) at smaller omissions, however, the current event model is preserved, and only changed parts are updated through inference processes. Thus far, reliably distinguishing those two processes in visual narratives like comics was difficult. We report a study (N = 80) that aimed to map the differences between constructing and updating event models in visual narratives by combining measures from narrative comprehension and event cognition research and manipulating event structure. Participants watched short visual narratives designed to (not) contain event boundaries at larger coherence breaks and elicit inferences through small omissions, while we collected viewing time measures as well as event segmentation and comprehensibility data. Viewing time, segmentation, and comprehensibility data were in line with the assumption of two distinct comprehension processes. We thus found converging evidence across multiple measures for distinct model construction and updating processes in visual narratives.

Short-term transfer effects of Tetris on mental rotation: Review and registered report - A Bayesian approach.

The existence of transfer effects of video games on cognitive performance are controversially discussed in experimental psychology. Whereas recent meta-analyses suggest the absence of far transfer effects, empirical evidence regarding near transfer effects is more controversial. This conceptual replication investigated the short-term near transfer effect of playing Tetris on mental rotation abilities. The design of the conceptual replication was based on a comprehensive compilation of the methods used by previous literature on this topic and advanced in order to reach a high scientific state-of-the-art standard. We ran a high-powered conceptual replication study with 366 participants randomly assigned to either an experimental group playing Tetris or a control group playing Solitaire. Both groups completed three commonly used mental rotation tests in a pre- and a posttest session. Additionally, the experimental group played Tetris while the control group played Solitaire. Playing time was 10 hours in total within 4 weeks. Based on previous research, we hypothesized that this might generate a short-term transfer effect of Tetris on mental rotation. While participants showed a repeated testing effect for the mental rotation tests in both groups, we found evidence that Tetris does not produce a short-term transfer effect on mental rotation. Both gender and expected outcomes did not influence this effect. Our study suggests that playing Tetris does not improve mental rotation skills.

Stories in the Mind? The Role of Story-Based Categorizations in Motion Classification.

Categorization is fundamental for spatial and motion representation in both the domain of artificial intelligence and human cognition. In this paper, we investigated whether motion categorizations designed in artificial intelligence can inform human cognition. More concretely, we investigated if such categorizations (also known as qualitative representations) can inform the psychological understanding of human perception and memory of motion scenes. To this end, we took two motion categorizations in artificial intelligence, Motion-RCC and Motion-OPRA1 , and conducted four experiments on human perception and memory. Participants viewed simple motion scenes and judged the similarity of transformed scenes with this reference scene. Those transformed scenes differed in none, one, or both Motion-RCC and Motion-OPRA1 categories. Importantly, we applied an equal absolute metric change to those transformed scenes, so that differences in the similarity judgments should be due only to differing categories. In Experiments 1a and 1b, where the reference stimulus and transformed stimuli were visible at the same time (perception), both Motion-OPRA1 and Motion-RCC influenced the similarity judgments, with a stronger influence of Motion-OPRA1 . In Experiments 2a and 2b, where participants first memorized the reference stimulus and viewed the transformed stimuli after a short blank (memory), only Motion-OPRA1 had marked influences on the similarity judgments. Our findings demonstrate a link between human cognition and these motion categorizations developed in artificial intelligence. We argue for a continued and close multidisciplinary approach to investigating the representation of motion scenes.

Mutual interplay between cognitive offloading and secondary task performance.

Various modern tools, such as smartphones, allow for cognitive offloading (i.e., the externalization of cognitive processes). In this study, we examined the use and consequences of cognitive offloading in demanding situations in which people perform multiple tasks concurrently-mimicking the requirements of daily life. In a preregistered study, we adapted the dual-task paradigm so that one of the tasks allowed for cognitive offloading. As a primary task, our participants (N = 172) performed the pattern copy task-a highly demanding working memory task that allows for offloading at various degrees. In this task, we manipulated the temporal costs of offloading. Concurrently, half of the participants responded to a secondary N-back task. As our main research question, we investigated the impact of offloading behavior on secondary task performance. We observed that more pronounced offloading in the condition without temporal costs was accompanied by a more accurate performance in the N-back task. Furthermore, the necessity to respond to the N-back task increased offloading behavior. These results suggest an interplay between offloading and secondary task performance: in demanding situations, individuals increasingly use cognitive offloading, which releases internal resources that can then be devoted to improving performance in other, concurrent tasks.

Processing spatial configurations in visuospatial working memory is influenced by shifts of overt visual attention.

When memorizing multiple objects, humans process them in relation to each other, proposing a configuration benefit. Shifts in overt visual attention through eye movements might influence the processing of spatial configurations. Whereas some research suggests that overt visual attention aids the processing of spatial representations, other research suggests a snapshot-like processing of spatial configurations, thus likely not relying on eye movements. In the first experiment, we focused on the comparison between an enforced fixation and a free view condition regarding configurational effects. Participants encoded objects' locations and were asked for changes at retrieval. One object was displaced in half of the trials and was either accompanied by a configuration or was displayed alone. In the second experiment, we expanded this idea by enforcing fixation during different task phases, namely encoding, maintenance and retrieval. We investigated if a fixed gaze during one specific phase drives the influence of eye movements when processing spatial configurations. We observed reliable configuration benefits for the free view conditions. Whereas a fixed gaze throughout the whole trial reduced the effect, enforced fixations during the task phases did not break the configuration benefit. Our findings suggest that whereas the processing of spatial configurations in memory is supported by the ability of performing shifts of overt visual attention, configurational processing does not rely on these shifts occurring throughout the task. Our results indicate a reciprocal relationship of visuospatial working memory and eye movements.

Long-term memory representations for audio-visual scenes.

In this study, we investigated the nature of long-term memory representations for naturalistic audio-visual scenes. Whereas previous research has shown that audio-visual scenes are recognized more accurately than their unimodal counterparts, it remains unclear whether this benefit stems from audio-visually integrated long-term memory representations or a summation of independent retrieval cues. We tested two predictions for audio-visually integrated memory representations. First, we used a modeling approach to test whether recognition performance for audio-visual scenes is more accurate than would be expected from independent retrieval cues. This analysis shows that audio-visual integration is not necessary to explain the benefit of audio-visual scenes relative to purely auditory or purely visual scenes. Second, we report a series of experiments investigating the occurrence of study-test congruency effects for unimodal and audio-visual scenes. Most importantly, visually encoded information was immune to additional auditory information presented during testing, whereas auditory encoded information was susceptible to additional visual information presented during testing. This renders a true integration of visual and auditory information in long-term memory representations unlikely. In sum, our results instead provide evidence for visual dominance in long-term memory. Whereas associative auditory information is capable of enhancing memory performance, the long-term memory representations appear to be primarily visual.

Framing cognitive offloading in terms of gains or losses: achieving a more optimal use of reminders.

Nowadays individuals can readily set reminders to offload intentions onto external resources, such as smartphone alerts, rather than using internal memory. Individuals tend to be biased, setting more reminders than would be optimal. We address the question whether the reminder bias depends on offloading scenarios being framed as either gains or losses, both between-participants (Experiment 1) and within-participants (Experiment 2). In both experiments, framing of reminders in terms of gains resulted in participants employing a risk-averse strategy and using more reminders than would be optimal. Importantly, however, participants used reminders more optimally and were more willing to choose the risk-seeking option of remembering internally when reminders implied a loss. Based on metacognitive measures in Experiment 2, the reminder bias increased the more underconfident participants were about their memory abilities in both framing scenarios. Framing did not alter this relationship between erroneous metacognitive underconfidence and reminder bias but provides an additional influence. We conclude that emphasizing the losses (costs) associated with external reminders helps in achieving more optimal decisions in offloading situations, and that in addition to cognitive effort and metacognitive judgments, framing needs to be considered in improving individuals' offloading behavior.

Individual differences in cognitive offloading: a comparison of intention offloading, pattern copy, and short-term memory capacity.

The cognitive load of many everyday life tasks exceeds known limitations of short-term memory. One strategy to compensate for information overload is cognitive offloading which refers to the externalization of cognitive processes such as reminder setting instead of memorizing. There appears to be remarkable variance in offloading behavior between participants which poses the question whether there is a common factor influencing offloading behavior across different tasks tackling short-term memory processes. To pursue this question, we studied individual differences in offloading behavior between two well-established offloading paradigms: the intention offloading task which tackles memory for intentions and the pattern copy task which tackles continuous short-term memory load. Our study also included an unrelated task measuring short-term memory capacity. Each participant completed all tasks twice on two consecutive days in order to obtain reliability scores. Despite high reliability scores, individual differences in offloading behavior were uncorrelated between the two offloading tasks. In both tasks, however, individual differences in offloading behavior were correlated with the individual differences in an unrelated short-term memory task. Our results therefore show that offloading behavior cannot simply be explained in terms of a single common factor driving offloading behavior across tasks. We discuss the implications of this finding for future research investigating the interrelations of offloading behavior across different tasks.

Consequences of cognitive offloading: Boosting performance but diminishing memory.

Modern technical tools such as tablets allow for the temporal externalisation of working memory processes (i.e., cognitive offloading). Although such externalisations support immediate performance on different tasks, little is known about potential long-term consequences of offloading behaviour. In the current set of experiments, we studied the relationship between cognitive offloading and subsequent memory for the offloaded information as well as the interplay of this relationship with the goal to acquire new memory representations. Our participants solved the Pattern Copy Task, in which we manipulated the costs of cognitive offloading and the awareness of a subsequent memory test. In Experiment 1 (N = 172), we showed that increasing the costs for offloading induces reduced offloading behaviour. This reduction in offloading came along with lower immediate task performance but more accurate memory in an unexpected test. In Experiment 2 (N = 172), we confirmed these findings and observed that offloading behaviour remained detrimental for subsequent memory performance when participants were aware of the upcoming memory test. Interestingly, Experiment 3 (N = 172) showed that cognitive offloading is not detrimental for long-term memory formation under all circumstances. Those participants who were forced to offload maximally but were aware of the memory test could almost completely counteract the negative impact of offloading on memory. Our experiments highlight the importance of the explicit goal to acquire new memory representations when relying on technical tools as offloading did have detrimental effects on memory without such a goal.

Do group ensemble statistics bias visual working memory for individual items? A registered replication of Brady and Alvarez (2011).

We performed a registered and precise replication of Experiment 1 reported in Brady and Alvarez (Psychological Science, 22, 384-392, 2011). The original experiment found that participants, who were asked to memorize the size of differently colored circles, reported the size of a probed circle biased toward the mean size of the same-colored group. Because our previous three unpublished replication attempts failed to find this effect, we powered the present registered replication using a Bayes Factor Design Analysis such that it provided compelling evidence regarding the presence or absence of the reported bias with a high probability, even under the assumption of smaller effect sizes. Thus, we recruited 663 participants through Amazon Mechanical Turk. We observed both a significant bias and strong Bayesian evidence in favor of the existence of a bias over the null hypothesis. Thus, our results can be considered a successful replication of the original findings, although with a considerably smaller effect size. We discuss the role of data quality when recruiting participants with Amazon Mechanical Turk. The present findings corroborate the idea that memory representations of individual objects are influenced by summary statistics.

From metacognitive beliefs to strategy selection: does fake performance feedback influence cognitive offloading?

The ubiquitous availability of technological aids requires individuals to constantly decide between either externalizing cognitive processes into these aids (i.e. cognitive offloading) or relying on their own internal cognitive resources. With the present research, we investigated the influence of metacognitive beliefs on individuals' offloading behavior in an experimental setup (N = 159). We manipulated participants' metacognitive beliefs about their memory abilities by providing fake performance feedback: below-average feedback, above-average feedback, or no feedback (control-group). We then measured offloading behavior, using a pattern copying task in which participants copied a color pattern from a model window into a workspace window. While solving this task, participants could rely either more on an internal memory strategy or more on an offloading strategy. Fake performance feedback affected the participants' metacognitive evaluations about their memory abilities (below-group < control-group < above-group). Although fake performance feedback did not affect actual offloading behavior, the participants receiving below-average performance feedback reported that they had relied more on an offloading strategy than those participants receiving above-average performance feedback. Furthermore, the participants in the below-group reported lower general memory abilities than the other groups at the end of the experiment. We conclude that while fake performance feedback strongly influenced metacognitive beliefs, this did not transfer into a change of strategy selection, thus not influencing offloading behavior. We propose to consider not only metacognitive beliefs but also metacognitive experiences as potential determinants of cognitive offloading.

(Re-)organisation of spatial configurations in visual working memory: The fate of objects rendered relevant or irrelevant by selective attention.

People maintain object locations not as independent absolute positions but based on inter-object relations in the form of a spatial configuration in visual working memory (VWM). Initial evidence suggests that spatial configurations can be reorganised during maintenance; however, this mechanism is not well understood. We report three experiments investigating this reorganisation process. First, we found that directing spatial attention with a retro cue triggers a reorganisation of spatial configurations during maintenance (Experiment 1). Second, we investigated the role of contextual objects rendered either relevant or irrelevant through the retro cue by manipulating the locations of the context at retrieval both within a partial display (Experiment 2a) and a whole display (Experiment 2b). Whereas the similar impairment of memory performance by changing the relevant and global context suggests a complete reorganisation of spatial configurations in VWM, this interpretation was challenged by the observation of impaired memory performance with changes of the irrelevant objects in a whole display that retains the relevant locations. Thus, we suggest that reorganisation should be considered to be the formation of a partial configuration based on the objects rendered relevant by the retro cue in addition to the originally encoded global configuration, with both configurations affecting memory performance.

Individual differences in visual attention: A short, reliable, open-source, and multilingual test of multiple object tracking in PsychoPy.

Individual differences in attentional abilities provide an interesting approach in studying visual attention as well as the relation of attention to other psychometric measures. However, recent research has demonstrated that many tasks from experimental research are not suitable for individual differences research, as they fail to capture these differences reliably. Here, we provide a test for individual differences in visual attention which relies on the multiple object tracking task (MOT). This test captures individual differences reliably in 6 to 15 min. Within the task, the participants have to maintain a set of targets (among identical distractors) across an interval of object motion. It captures the efficiency of attentional deployment. Importantly, this test was explicitly designed and tested for reliability under conditions that match those of most laboratory research (restricted sample of students, approximately n = 50). The test is free to use and runs fully under open-source software. In order to facilitate the application of the test, we have translated it into 16 common languages (Chinese, Danish, Dutch, English, Finnish, French, German, Italian, Japanese, Norwegian, Polish, Portuguese, Russian, Spanish, Swedish, and Turkish). The test can be downloaded at https://osf.io/qy6nb/ . We hope that this MOT test supports researchers whose field of study requires capturing individual differences in visual attention reliably.

Cross-codal integration of bridging-event information in narrative understanding.

Visual narratives communicate event sequences by using different code systems such as pictures and texts. Thus, comprehenders must integrate information from different codalities. This study addressed such cross-codal integration processes by investigating how the codality of bridging-event information (i.e., pictures, text) affects the understanding of visual narrative events. In Experiment 1, bridging-event information was either present (as picture or text) or absent (i.e., not shown). The viewing times for the subsequent picture depicting the end state of the action were comparable within the absent and the text conditions. Further, the viewing times for the end-state picture were significantly longer in the text condition as compared to the pictorial condition. In Experiment 2, we tested whether replacing bridging-event information with a blank panel increases viewing times in a way similar to the text condition. Bridging event information was either present (as picture) or absent (not shown vs. blank panel). The results replicated Experiment 1. Additionally, the viewing times for the end-state pictures were longest in the blank condition. In Experiment 3, we investigated the costs related to integrating information from different codalities by directly comparing the text and picture conditions with the blank condition. The results showed that the distortion caused by the blank panel is larger than the distortion caused by cross-codal integration processes. Summarizing, we conclude that cross-codal information processing during narrative comprehension is possible but associated with additional mental effort. We discuss the results with regard to theories of narrative understanding.

Reorganization of spatial configurations in visual working memory: A matter of set size?

Humans process single objects in relation to other simultaneously maintained objects in visual working memory. This interdependence is called spatial configuration. Humans are able to reorganize global spatial configurations into relevant partial configurations. We conducted three experiments investigating the process underlying reorganization by manipulating memory set size and the presence of configurations at retrieval. Participants performed a location change detection task for a single object probed at retrieval. At the beginning of each trial, participants memorized the locations of all objects (set size: 4, 8, 12, or 16). During maintenance, a valid retro cue highlighted the side containing the object probed at retrieval, thus enabling participants to reorganize the memorized global spatial configuration to the partial cued configuration. At retrieval, the object probed was shown together with either all objects (complete configuration; Experiment 1a), the cued objects only (congruent configuration; all Experiments), the non-cued objects only (incongruent configuration, all Experiments) or alone (no configuration; Experiment 1b). We observed reorganization of spatial configurations as indicated by a superior location change detection performance with a congruent partial configuration than an incongruent partial configuration across all three experiments. We also observed an overall decrease in accuracy with increasing set size. Most importantly, however, we did not find evidence for a reliable impairment of reorganization with increasing set size. We discuss these findings with regard to the memory representation underlying spatial configurations.

Reorganization of spatial configurations in visual working memory.

Human beings have to constantly process multiple objects in visual working memory (VWM). Positional relations to other objects known as spatial configurations contribute significantly to the organization of information in VWM. The aim of our study was to clarify whether spatial configurations can be reorganized to a subset of objects during maintenance. Participants were shown an array of objects, and afterwards the objects disappeared. A valid cue was shown either during encoding or maintenance to highlight the side of the following probed object. Afterwards, the objects reappeared and participants were instructed to detect whether or not a particular object changed its location. We manipulated the configurations at retrieval regarding the number of objects, ranging from all objects to a single object. Our first and second experiment investigated reorganization for a number of six and 12 objects, respectively. In the third experiment, we used a retro cue only and manipulated eye movements (free view vs. enforced fixation). While showing that reorganization of spatial configurations during maintenance is possible in principle, we found some boundary conditions. There was no spatial configuration effect when participants had to fixate. Thus, eye movements are required for a configuration effect to occur.

Filling the gap despite full attention: the role of fast backward inferences for event completion.

The comprehension of dynamic naturalistic events poses at least two challenges to the cognitive system: filtering relevant information with attention and dealing with information that was missing or missed. With four experiments, we studied the completion of missing information despite full attention. Participants watched short soccer video clips and we informed participants that we removed a critical moment of ball contact in half of the clips. We asked participants to detect whether these moments of ball contact were present or absent. In Experiment 1, participants gave their detection responses either directly during an event or delayed after an event. Although participants directed their full attention toward the critical contact moment, they were more likely to indicate seeing the missing ball contact if it was followed by a causally matching scene than if it was followed by an unrelated scene, both for the immediate and delayed responses. Thus, event completion occurs quickly. In Experiment 2, only a causally matching scene but neither a white mask nor an irrelevant scene caused the completion of missing information. This indicates that the completion of missing information is caused by backward inferences rather than predictive perception. In Experiment 3, we showed that event completion occurs directly during a trial and does not depend on expectations built up after seeing the same causality condition multiple times. In Experiment 4, we linked our findings to event cognition by asking participants to perform a natural segmentation task. We conclude that observers complete missing information during coherent events based on a fast backward inference mechanism even when directing their attention toward the missing information.

Linguistic Information in Auditory Dynamic Events Contributes to the Detection of Fine, Not Coarse Event Boundaries.

Human observers (comprehenders) segment dynamic information into discrete events. That is, although there is continuous sensory information, comprehenders perceive boundaries between two meaningful units of information. In narrative comprehension, comprehenders use linguistic, non-linguistic , and physical cues for this event boundary perception. Yet, it is an open question - both from a theoretical and an empirical perspective - how linguistic and non-linguistic cues contribute to this process. The current study explores how linguistic cues contribute to the participants' ability to segment continuous auditory information into discrete, hierarchically structured events. Native speakers of German and non-native speakers, who neither spoke nor understood German, segmented a German audio drama into coarse and fine events. Whereas native participants could make use of linguistic, non-linguistic, and physical cues for segmentation, non-native participants could only use non-linguistic and physical cues. We analyzed segmentation behavior in terms of the ability to identify coarse and fine event boundaries and the resulting hierarchical structure. Non-native listeners identified almost identical coarse event boundaries as native listeners, but missed some of the fine event boundaries identified by the native listeners. Interestingly, hierarchical event perception (as measured by hierarchical alignment and enclosure) was comparable for native and non-native participants. In summary, linguistic cues contributed particularly to the identification of certain fine event boundaries. The results are discussed with regard to the current theories of event cognition.

Action goal changes caused by agents and patients both induce global updating of event models.

Observers represent everyday actions in event models along multiple dimensions such as space, time, or goals. Whenever new information along those dimensions is perceived, the event model is updated accordingly. In 3 experiments, we investigated event model updating associated with goal changes during ongoing actions that involved both an agent performing the action and a patient receiving the action. We presented short action clips showing goal-directed actions-such as handing over a book-as self-paced slideshows. Those action sequences contained either no goal change or a goal change caused by either the agent or the patient of the action. We measured viewing times (Experiments 1-3) and verbal action descriptions (Experiment 3). The action descriptions revealed that patients causing a goal change turned into agents in the event model. Despite the updating of the additional dimension (protagonist) for goal changes caused by patients, goal changes caused by agents and patients were associated with a similar updating effort as indicated by the viewing times and a similar increase in the complexity of the represented event structure as indicated by the number of clauses used in the action descriptions. Thus, updating was global rather than incremental in our experiments. We conclude that goal changes cause a global updating of event models independent of the source of the goal change. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Construction and updating of event models in auditory event processing.

Humans segment the continuous stream of sensory information into distinct events at points of change. Between 2 events, humans perceive an event boundary. Present theories propose changes in the sensory information to trigger updating processes of the present event model. Increased encoding effort finally leads to a memory benefit at event boundaries. Evidence from reading time studies (increased reading times with increasing amount of change) suggest that updating of event models is incremental. We present results from 5 experiments that studied event processing (including memory formation processes and reading times) using an audio drama as well as a transcript thereof as stimulus material. Experiments 1a and 1b replicated the event boundary advantage effect for memory. In contrast to recent evidence from studies using visual stimulus material, Experiments 2a and 2b found no support for incremental updating with normally sighted and blind participants for recognition memory. In Experiment 3, we replicated Experiment 2a using a written transcript of the audio drama as stimulus material, allowing us to disentangle encoding and retrieval processes. Our results indicate incremental updating processes at encoding (as measured with reading times). At the same time, we again found recognition performance to be unaffected by the amount of change. We discuss these findings in light of current event cognition theories. (PsycINFO Database Record