Integration or separation? Spatial and temporal representations of whole-body movements in visual working memory.

Spatial and temporal information are two major feature dimensions of human movements. How these two types of information are represented in working memory-whether as integrated units or as individual features-influences how much information might be retained and how the retained information might be manipulated. In this study, we investigated how spatial (path/trajectory) and temporal (speed/rhythm) information of complex whole-body movements are represented in working memory under a more ecologically valid condition wherein the spatiotemporal continuity of movement sequences was considered. We found that the spatial and temporal information are not automatically integrated but share the storage capacity and compete for a common pool of cognitive resources. The finding rejects the strong form of object-based representation and supports the partial independence of spatial and temporal processing. Nevertheless, we also found that contextual factors, such as the way movements are organized and displayed, can further modulate the level of object-based representation and spatiotemporal integration.

Effects of spatiotemporal (dis)continuity on working memory for human movements.

Human movements are dynamic and continuous in nature. However, how the spatiotemporal continuity influences working memory for movements is still unclear. Specifically, spatiotemporal continuity of movements may facilitate integrative processing ("integration") and enhance memory performance by optimizing the encoding process, but it may also diminish memory benefits from distinctive processing ("separation"). In this study, we manipulated the continuity state (continuous/discontinuous) (Experiment 1) and its predictability (Experiment 2) of whole-body movement sequences and tested participants' working memory for observed movements with a single-probe recognition task. We formulated potential influence from spatiotemporal (dis)continuity by two opposite forces - integration vs. separation, and demonstrated a conflict between these two processes across space and time. Moreover, we found that the seemingly stimulus-driven perceptual effects from spatiotemporal (dis)continuity might be supported by a prediction-based mechanism, which guided the selection of an optimal processing strategy. Overall, our finding illustrates an interweaving relationship between spatial and temporal processing during action observation and highlights the importance of considering the dynamic and continuous nature of human movements in visual perception and working memory research.

Attention modulates incidental memory encoding of human movements.

Attention has been shown to enhance the processing of task-relevant information while suppressing the processing of task-irrelevant information. However, it is less clear whether this attentional modulation exists when there is an intrinsic dependence between task-relevant and task-irrelevant information, such as the dependence of temporal processing on spatial information. In this study, we used complex whole-body movement sequences to investigate the extent to which the task-irrelevant spatial information (trajectory) is processed when only the temporal information (rhythm) is in focus. Moreover, we examined, if the task-irrelevant spatial information is "co-selected" with the target temporal information as predicted by the intrinsic spatiotemporal dependence, whether task-driven attention that is actively directed to spatial information provides extra benefits. Through a two-phase experiment (an incidental encoding phase followed by a surprise memory test phase), we found that the task-irrelevant spatial information was not only perceived but also encoded in memory, providing further evidence in support of a relatively automatic co-selection of spatial information in temporal processing. Nevertheless, we also found that movements whose trajectories were intentionally attended to during the encoding phase were recognized better in the test phase than those that were not, indicating a further modulation from attention on incidental memory encoding and information processing.

Action Selection and Execution in Everyday Activities: A Cognitive Robotics and Situation Model Perspective.

We examine the mechanisms required to handle everyday activities from the standpoint of cognitive robotics, distinguishing activities on the basis of complexity and transparency. Task complexity (simple or complex) reflects the intrinsic nature of a task, while task transparency (easy or difficult) reflects an agent's ability to identify a solution strategy in a given task. We show how the CRAM cognitive architecture allows a robot to carry out simple and complex activities such as laying a table for a meal and loading a dishwasher afterward. It achieves this by using generalized action plans that exploit reasoning with modular, composable knowledge chunks representing general knowledge to transform underdetermined everyday action requests into motion plans that successfully accomplish the required task. Noting that CRAM does not yet have the ability to deal with difficult activities, we leverage insights from the situation model perspective on the cognitive mechanisms underlying flexible context-sensitive behavior with a view to extending CRAM to overcome this deficit.

Bimanual Coordination Learning with Different Augmented Feedback Modalities and Information Types.

Previous studies have shown that bimanual coordination learning is more resistant to the removal of augmented feedback when acquired with auditory than with visual channel. However, it is unclear whether this differential "guidance effect" between feedback modalities is due to enhanced sensorimotor integration via the non-dominant auditory channel or strengthened linkage to kinesthetic information under rhythmic input. The current study aimed to examine how modalities (visual vs. auditory) and information types (continuous visuospatial vs. discrete rhythmic) of concurrent augmented feedback influence bimanual coordination learning. Participants either learned a 90°-out-of-phase pattern for three consecutive days with Lissajous feedback indicating the integrated position of both arms, or with visual or auditory rhythmic feedback reflecting the relative timing of the movement. The results showed diverse performance change after practice when the feedback was removed between Lissajous and the other two rhythmic groups, indicating that the guidance effect may be modulated by the type of information provided during practice. Moreover, significant performance improvement in the dual-task condition where the irregular rhythm counting task was applied as a secondary task also suggested that lower involvement of conscious control may result in better performance in bimanual coordination.