Fast Hand Movements Unveil Multifractal Roots of Adaptation in the Visuomotor Cognitive System.

Beyond apparent simplicity, visuomotor dexterity actually requires the coordination of multiple interactions across a complex system that links the brain, the body and the environment. Recent research suggests that a better understanding of how perceptive, cognitive and motor activities cohere to form executive control could be gained from multifractal formalisms applied to movement behavior. Rather than a central executive "talking" to encapsuled components, the multifractal intuition suggests that eye-hand coordination arises from multiplicative cascade dynamics across temporal scales of activity within the whole system, which is reflected in movement time series. Here we examined hand movements of sport students performing a visuomotor task in virtual reality (VR). The task involved hitting spatially arranged targets that lit up on a virtual board under critical time pressure. Three conditions were compared where the visual search field changed: whole board (Standard), half-board lower view field (LVF) and upper view field (UVF). Densely sampled (90 Hz) time series of hand motions captured by VR controllers were analyzed by a focus-based multifractal detrended fluctuation analysis (DFA). Multiplicative rather than additive interactions across temporal scales were evidenced by testing comparatively phase-randomized surrogates of experimental series, which confirmed nonlinear processes. As main results, it was demonstrated that: (i) the degree of multifractality in hand motion behavior was minimal in LVF, a familiar visual search field where subjects correlatively reached their best visuomotor response times (RTs); (ii) multifractality increased in the less familiar UVF, but interestingly only for the non-dominant hand; and (iii) multifractality increased further in Standard, for both hands indifferently; in Standard, the maximal expansion of the visual search field imposed the highest demand as evidenced by the worst visuomotor RTs. Our observations advocate for visuomotor dexterity best described by multiplicative cascades dynamics and a system-wide distributed control rather than a central executive. More importantly, multifractal metrics obtained from hand movements behavior, beyond the confines of the brain, offer a window on the fine organization of control architecture, with high sensitivity to hand-related control behavior under specific constraints. Appealing applications may be found in movement learning/rehabilitation, e.g., in hemineglect people, stroke patients, maturing children or athletes.

Reliability of the Dynavision task in virtual reality to explore visuomotor phenotypes.

Daily-life behaviors strongly rely on visuomotor integration, a complex sensorimotor process with obvious plasticity. Visual-perceptive and visual-cognitive functions are degraded by neurological disorders and brain damage, but are improved by vision training, e.g. in athletes. Hence, developing tools to evaluate/improve visuomotor abilities has found echo among psychologists, neurophysiologists, clinicians and sport professionals. Here we implemented the Dynavision visuomotor reaction task in virtual reality (VR) to get a flexible tool to place high demands on visual-perceptive and visual-cognitive processes, and explore individual abilities in visuomotor integration. First, we demonstrated high test-retest reliability for the task in VR among healthy physically-active students (n = 64, 32 females). Second, the capture of head movements thanks to the VR-headset sensors provided new and reliable information on individual visual-perceptual strategies, which added significant value to explore visuomotor phenotypes. A factor analysis of mixed data and hierarchical clustering on principal components points to head movements, video-games practice and ball-tracking sports as critical cues to draw visuomotor phenotypes among our participants. We conclude that the visuomotor task in VR is a reliable, flexible and promising tool. Since VR nowadays can serve e.g. to modulate multisensorial integration by creating visual interoceptive-exteroceptive conflicts, or placing specifically designed cognitive demand, much could be learned on complex integrated visuomotor processes through VR experiments. This offers new perspectives for post brain injury risk evaluation, rehabilitation programs and visual-cognitive training.

Flaws in current human training protocols for spontaneous Brain-Computer Interfaces: lessons learned from instructional design.

While recent research on Brain-Computer Interfaces (BCI) has highlighted their potential for many applications, they remain barely used outside laboratories. The main reason is their lack of robustness. Indeed, with current BCI, mental state recognition is usually slow and often incorrect. Spontaneous BCI (i.e., mental imagery-based BCI) often rely on mutual learning efforts by the user and the machine, with BCI users learning to produce stable ElectroEncephaloGraphy (EEG) patterns (spontaneous BCI control being widely acknowledged as a skill) while the computer learns to automatically recognize these EEG patterns, using signal processing. Most research so far was focused on signal processing, mostly neglecting the human in the loop. However, how well the user masters the BCI skill is also a key element explaining BCI robustness. Indeed, if the user is not able to produce stable and distinct EEG patterns, then no signal processing algorithm would be able to recognize them. Unfortunately, despite the importance of BCI training protocols, they have been scarcely studied so far, and used mostly unchanged for years. In this paper, we advocate that current human training approaches for spontaneous BCI are most likely inappropriate. We notably study instructional design literature in order to identify the key requirements and guidelines for a successful training procedure that promotes a good and efficient skill learning. This literature study highlights that current spontaneous BCI user training procedures satisfy very few of these requirements and hence are likely to be suboptimal. We therefore identify the flaws in BCI training protocols according to instructional design principles, at several levels: in the instructions provided to the user, in the tasks he/she has to perform, and in the feedback provided. For each level, we propose new research directions that are theoretically expected to address some of these flaws and to help users learn the BCI skill more efficiently.

Age-related wayfinding differences in real large-scale environments: detrimental motor control effects during spatial learning are mediated by executive decline?

The aim of this study was to evaluate motor control activity (active vs. passive condition) with regards to wayfinding and spatial learning difficulties in large-scale spaces for older adults. We compared virtual reality (VR)-based wayfinding and spatial memory (survey and route knowledge) performances between 30 younger and 30 older adults. A significant effect of age was obtained on the wayfinding performances but not on the spatial memory performances. Specifically, the active condition deteriorated the survey measure in all of the participants and increased the age-related differences in the wayfinding performances. Importantly, the age-related differences in the wayfinding performances, after an active condition, were further mediated by the executive measures. All of the results relative to a detrimental effect of motor activity are discussed in terms of a dual task effect as well as executive decline associated with aging.

Executive and memory correlates of age-related differences in wayfinding performances using a virtual reality application.

The aim of this study was to evaluate in large-scale spaces wayfinding and spatial learning difficulties for older adults in relation to the executive and memory decline associated with aging. We compared virtual reality (VR)-based wayfinding and spatial memory performances between young and older adults. Wayfinding and spatial memory performances were correlated with classical measures of executive and visuo-spatial memory functions, but also with self-reported estimates of wayfinding difficulties. We obtained a significant effect of age on wayfinding performances but not on spatial memory performances. The overall correlations showed significant correlations between the wayfinding performances and the classical measures of both executive and visuo-spatial memory, but only when the age factor was not partialled out. Also, older adults underestimated their wayfinding difficulties. A significant relationship between the wayfinding performances and self-reported wayfinding difficulty estimates is found, but only when the age effect was partialled out. These results show that, even when older adults have an equivalent spatial knowledge to young adults, they had greater difficulties with the wayfinding task, supporting an executive decline view in age-related wayfinding difficulties. However, the correlation results are in favor of both the memory and executive decline views as mediators of age-related differences in wayfinding performances. This is discussed in terms of the relationships between memory and executive functioning in wayfinding task orchestration. Our results also favor the use of objective assessments of everyday navigation difficulties in virtual applications, instead of self-reported questionnaires, since older adults showed difficulties in estimating their everyday wayfinding problems.

Do patients with traumatic brain injury learn a route in the same way in real and virtual environments?

UNLABELLED: An increasing number of studies address the use of virtual environments (VE) in the cognitive assessment of spatial abilities. However, the differences between learning in a VE and a real environment (RE) remain controversial. PURPOSE: To compare the topographical behavior and spatial representations of patients with traumatic brain injury navigating in a real environment and in a virtual reproduction of this environment. METHODS: Twenty-seven subjects with moderate to severe traumatic brain injury were consecutively included and allocated to one of two groups. The subjects were taught the same route in either the virtual environment or the real environment and had to recall it twice immediately after learning the route and once after a delay. At the end of these sessions, the subjects were asked to complete three representational tests: a map test, a map recognition test recognition and a scene arrangement test. RESULTS: No significant difference was found between the two groups with regards to demographics, severity of brain injury or episodic memory. As a main result, the number of error rates did not significantly differ between the real and virtual environment [F (1, 25) = 0.679; p = 0.4176)]. Scores on the scene arrangement test were higher in the real environment [U = 32.5; p = 0.01]. CONCLUSIONS: Although spatial representations probably differ between the real and virtual environment, virtual reality remains a trusty assessment tool for spatial abilities.

The use of virtual reality for episodic memory assessment: effects of active navigation.

Episodic memory was assessed using Virtual Reality (VR). Forty-four (44) subjects visualized a target virtual apartment containing specific objects in each room. Then they visualized a second virtual apartment comprised of specific objects and objects shared by the two apartments. Subjects navigated in the virtual apartments in one of the following two conditions: active and passive. Four main episodic memory components were scored from the VR exposures: (1) learning effect; (2) active forgetting effect; (3) strategies at encoding and at retrieval; and (4) false recognitions (FRs). The effect of navigation mode (active vs. passive) on each memory component was examined. Active subjects had better learning and retrieval (recognition hits) performances compared to passive subjects. A beneficial effect of active navigation was also observed on the source-based FR rates. Active subjects made fewer source-based FRs compared to passive subjects. These overall results for the effect of active navigation are discussed in terms of the distinction between item-specific and relational processing.

Virtual/real transfer of spatial knowledge: benefit from visual fidelity provided in a virtual environment and impact of active navigation.

The purpose of this study was to evaluate the effect the visual fidelity of a virtual environment (VE) (undetailed vs. detailed) has on the transfer of spatial knowledge based on the navigation mode (passive vs. active) for three different spatial recall tasks (wayfinding, sketch mapping, and picture sorting). Sixty-four subjects (32 men and 32 women) participated in the experiment. Spatial learning was evaluated by these three tasks in the context of the Bordeaux district. In the wayfinding task, the results indicated that the detailed VE helped subjects to transfer their spatial knowledge from the VE to the real world, irrespective of the navigation mode. In the sketch-mapping task, the detailed VE increased performances compared to the undetailed VE condition, and allowed subjects to benefit from the active navigation. In the sorting task, performances were better in the detailed VE; however, in the undetailed version of the VE, active learning either did not help the subjects or it even deteriorated their performances. These results are discussed in terms of appropriate perceptive-motor and/or spatial representations for each spatial recall task.

Virtual/Real transfer of spatial learning: impact of activity according to the retention delay.

Within the framework of cognitive rehabilitation using virtual reality (VR), one of the major challenges is to study beforehand the effectiveness of the virtual-real transfer of learning and to define cognitive aids. The aim of this experiment was to verify if, after learning spatial knowledge (i.e., a route) in VR, performances can be transferred to reality, then maintained in real time, and improved with the aid of an active navigation (i.e., using a joystick). Ninety student volunteers from the University of Bordeaux 2 (45 men and 45 women) participated in the experiment. The virtual environment (VE) used for learning was a replica of an area of Bordeaux. The factors tested were retention delay (Immediate vs. 48 hours) and type of navigation (Passive virtual vs. Active virtual vs. Real), using three recall tasks: wayfinding, freehand sketch and photograph classification. Our results showed that the virtual-real transfer was not degraded by a retention delay of 48 hours and that active navigation allowed performances to be optimized.