Psychophysiological Markers of Performance and Learning during Simulated Marksmanship in Immersive Virtual Reality.

The fusion of immersive virtual reality, kinematic movement tracking, and EEG offers a powerful test bed for naturalistic neuroscience research. Here, we combined these elements to investigate the neuro-behavioral mechanisms underlying precision visual-motor control as 20 participants completed a three-visit, visual-motor, coincidence-anticipation task, modeled after Olympic Trap Shooting and performed in immersive and interactive virtual reality. Analyses of the kinematic metrics demonstrated learning of more efficient movements with significantly faster hand RTs, earlier trigger response times, and higher spatial precision, leading to an average of 13% improvement in shot scores across the visits. As revealed through spectral and time-locked analyses of the EEG beta band (13-30 Hz), power measured prior to target launch and visual-evoked potential amplitudes measured immediately after the target launch correlated with subsequent reactive kinematic performance in the shooting task. Moreover, both launch-locked and shot/feedback-locked visual-evoked potentials became earlier and more negative with practice, pointing to neural mechanisms that may contribute to the development of visual-motor proficiency. Collectively, these findings illustrate EEG and kinematic biomarkers of precision motor control and changes in the neurophysiological substrates that may underlie motor learning.

Measuring visual search and distraction in immersive virtual reality.

An important issue of psychological research is how experiments conducted in the laboratory or theories based on such experiments relate to human performance in daily life. Immersive virtual reality (VR) allows control over stimuli and conditions at increased ecological validity. The goal of the present study was to accomplish a transfer of traditional paradigms that assess attention and distraction to immersive VR. To further increase ecological validity we explored attentional effects with daily objects as stimuli instead of simple letters. Participants searched for a target among distractors on the countertop of a virtual kitchen. Target-distractor discriminability was varied and the displays were accompanied by a peripheral flanker that was congruent or incongruent to the target. Reaction time was slower when target-distractor discriminability was low and when flankers were incongruent. The results were replicated in a second experiment in which stimuli were presented on a computer screen in two dimensions. The study demonstrates the successful translation of traditional paradigms and manipulations into immersive VR and lays a foundation for future research on attention and distraction in VR. Further, we provide an outline for future studies that should use features of VR that are not available in traditional laboratory research.

Sensorimotor Learning during a Marksmanship Task in Immersive Virtual Reality.

Sensorimotor learning refers to improvements that occur through practice in the performance of sensory-guided motor behaviors. Leveraging novel technical capabilities of an immersive virtual environment, we probed the component kinematic processes that mediate sensorimotor learning. Twenty naïve subjects performed a simulated marksmanship task modeled after Olympic Trap Shooting standards. We measured movement kinematics and shooting performance as participants practiced 350 trials while receiving trial-by-trial feedback about shooting success. Spatiotemporal analysis of motion tracking elucidated the ballistic and refinement phases of hand movements. We found systematic changes in movement kinematics that accompanied improvements in shot accuracy during training, though reaction and response times did not change over blocks. In particular, we observed longer, slower, and more precise ballistic movements that replaced effort spent on corrections and refinement. Collectively, these results leverage developments in immersive virtual reality technology to quantify and compare the kinematics of movement during early learning of full-body sensorimotor orienting.

Wireless, Web-Based Interactive Control of Optical Coherence Tomography with Mobile Devices.

PURPOSE: Optical coherence tomography (OCT) is widely used in ophthalmology clinics and has potential for more general medical settings and remote diagnostics. In anticipation of remote applications, we developed wireless interactive control of an OCT system using mobile devices. METHODS: A web-based user interface (WebUI) was developed to interact with a handheld OCT system. The WebUI consisted of key OCT displays and controls ported to a webpage using HTML and JavaScript. Client-server relationships were created between the WebUI and the OCT system computer. The WebUI was accessed on a cellular phone mounted to the handheld OCT probe to wirelessly control the OCT system. Twenty subjects were imaged using the WebUI to assess the system. System latency was measured using different connection types (wireless 802.11n only, wireless to remote virtual private network [VPN], and cellular). RESULTS: Using a cellular phone, the WebUI was successfully used to capture posterior eye OCT images in all subjects. Simultaneous interactivity by a remote user on a laptop was also demonstrated. On average, use of the WebUI added only 58, 95, and 170 ms to the system latency using wireless only, wireless to VPN, and cellular connections, respectively. Qualitatively, operator usage was not affected. CONCLUSIONS: Using a WebUI, we demonstrated wireless and remote control of an OCT system with mobile devices. TRANSLATIONAL RELEVANCE: The web and open source software tools used in this project make it possible for any mobile device to potentially control an OCT system through a WebUI. This platform can be a basis for remote, teleophthalmology applications using OCT.

Medial prefrontal pathways for the contextual regulation of extinguished fear in humans.

The maintenance of anxiety disorders is thought to depend, in part, on deficits in extinction memory, possibly due to reduced contextual control of extinction that leads to fear renewal. Animal studies suggest that the neural circuitry responsible fear renewal includes the hippocampus, amygdala, and dorsomedial (dmPFC) and ventromedial (vmPFC) prefrontal cortex. However, the neural mechanisms of context-dependent fear renewal in humans remain poorly understood. We used functional magnetic resonance imaging (fMRI), combined with psychophysiology and immersive virtual reality, to elucidate how the hippocampus, amygdala, and dmPFC and vmPFC interact to drive the context-dependent renewal of extinguished fear. Healthy human participants encountered dynamic fear-relevant conditioned stimuli (CSs) while navigating through 3-D virtual reality environments in the MRI scanner. Conditioning and extinction were performed in two different virtual contexts. Twenty-four hours later, participants were exposed to the CSs without reinforcement while navigating through both contexts in the MRI scanner. Participants showed enhanced skin conductance responses (SCRs) to the previously-reinforced CS+ in the acquisition context on Day 2, consistent with fear renewal, and sustained responses in the dmPFC. In contrast, participants showed low SCRs to the CSs in the extinction context on Day 2, consistent with extinction recall, and enhanced vmPFC activation to the non-reinforced CS-. Structural equation modeling revealed that the dmPFC fully mediated the effect of the hippocampus on right amygdala activity during fear renewal, whereas the vmPFC partially mediated the effect of the hippocampus on right amygdala activity during extinction recall. These results indicate dissociable contextual influences of the hippocampus on prefrontal pathways, which, in turn, determine the level of reactivation of fear associations.

Extinction in multiple virtual reality contexts diminishes fear reinstatement in humans.

Although conditioned fear can be effectively extinguished by unreinforced exposure to a threat cue, fear responses tend to return when the cue is encountered some time after extinction (spontaneous recovery), in a novel environment (renewal), or following presentation of an aversive stimulus (reinstatement). As extinction represents a context-dependent form of new learning, one possible strategy to circumvent the return of fear is to conduct extinction across several environments. Here, we tested the effectiveness of multiple context extinction in a two-day fear conditioning experiment using 3-D virtual reality technology to create immersive, ecologically-valid context changes. Fear-potentiated startle served as the dependent measure. All three experimental groups initially acquired fear in a single context. A multiple extinction group then underwent extinction in three contexts, while a second group underwent extinction in the acquisition context and a third group underwent extinction in a single different context. All groups returned 24h later to test for return of fear in the extinction context (spontaneous recovery) and a novel context (renewal and reinstatement/test). Extinction in multiple contexts attenuated reinstatement of fear but did not reduce spontaneous recovery. Results from fear renewal were tendential. Our findings suggest that multi-context extinction can reduce fear relapse following an aversive event--an event that often induces return of fear in real-world settings--and provides empirical support for conducting exposure-based clinical treatments across a variety of environments.

Effects of virtual environment platforms on emotional responses.

The goal of the current study was to investigate the effects of different virtual environment (VE) technologies (i.e., desktop, head mounted display, or fully immersive platforms) on emotional arousal and task performance. Fifty-three participants were recruited from a college population. Reactivity to stressful VEs was examined in three VE systems from desktop to high-end fully immersive systems. The experiment was a 3 (desktop system, head mounted display, and six wall system)×2 (high- and low-stressful VE) within subject design, with self-reported emotional arousal and valence, skin conductance, task performance, presence, and simulator sickness examined as dependent variables. Replicating previous studies, the fully immersive system induced the highest sense of presence and the head mounted display system elicited the highest amount of simulator sickness. Extending previous studies, the results demonstrated that VE platforms were associated with different patterns in emotional responses and task performance. Our findings suggest that different VE systems may be appropriate for different scientific purposes when studying stress reactivity using emotionally evocative tasks.

Evaluating display fidelity and interaction fidelity in a virtual reality game.

In recent years, consumers have witnessed a technological revolution that has delivered more-realistic experiences in their own homes through high-definition, stereoscopic televisions and natural, gesture-based video game consoles. Although these experiences are more realistic, offering higher levels of fidelity, it is not clear how the increased display and interaction aspects of fidelity impact the user experience. Since immersive virtual reality (VR) allows us to achieve very high levels of fidelity, we designed and conducted a study that used a six-sided CAVE to evaluate display fidelity and interaction fidelity independently, at extremely high and low levels, for a VR first-person shooter (FPS) game. Our goal was to gain a better understanding of the effects of fidelity on the user in a complex, performance-intensive context. The results of our study indicate that both display and interaction fidelity significantly affect strategy and performance, as well as subjective judgments of presence, engagement, and usability. In particular, performance results were strongly in favor of two conditions: low-display, low-interaction fidelity (representative of traditional FPS games) and high-display, high-interaction fidelity (similar to the real world).

Revealing context-specific conditioned fear memories with full immersion virtual reality.

The extinction of conditioned fear is known to be context-specific and is often considered more contextually bound than the fear memory itself (Bouton, 2004). Yet, recent findings in rodents have challenged the notion that contextual fear retention is initially generalized. The context-specificity of a cued fear memory to the learning context has not been addressed in the human literature largely due to limitations in methodology. Here we adapt a novel technology to test the context-specificity of cued fear conditioning using full immersion 3-D virtual reality (VR). During acquisition training, healthy participants navigated through virtual environments containing dynamic snake and spider conditioned stimuli (CSs), one of which was paired with electrical wrist stimulation. During a 24-h delayed retention test, one group returned to the same context as acquisition training whereas another group experienced the CSs in a novel context. Unconditioned stimulus expectancy ratings were assayed on-line during fear acquisition as an index of contingency awareness. Skin conductance responses time-locked to CS onset were the dependent measure of cued fear, and skin conductance levels during the interstimulus interval were an index of context fear. Findings indicate that early in acquisition training, participants express contingency awareness as well as differential contextual fear, whereas differential cued fear emerged later in acquisition. During the retention test, differential cued fear retention was enhanced in the group who returned to the same context as acquisition training relative to the context shift group. The results extend recent rodent work to illustrate differences in cued and context fear acquisition and the contextual specificity of recent fear memories. Findings support the use of full immersion VR as a novel tool in cognitive neuroscience to bridge rodent models of contextual phenomena underlying human clinical disorders.

KinImmerse: Macromolecular VR for NMR ensembles.

BACKGROUND: In molecular applications, virtual reality (VR) and immersive virtual environments have generally been used and valued for the visual and interactive experience - to enhance intuition and communicate excitement - rather than as part of the actual research process. In contrast, this work develops a software infrastructure for research use and illustrates such use on a specific case. METHODS: The Syzygy open-source toolkit for VR software was used to write the KinImmerse program, which translates the molecular capabilities of the kinemage graphics format into software for display and manipulation in the DiVE (Duke immersive Virtual Environment) or other VR system. KinImmerse is supported by the flexible display construction and editing features in the KiNG kinemage viewer and it implements new forms of user interaction in the DiVE. RESULTS: In addition to molecular visualizations and navigation, KinImmerse provides a set of research tools for manipulation, identification, co-centering of multiple models, free-form 3D annotation, and output of results. The molecular research test case analyzes the local neighborhood around an individual atom within an ensemble of nuclear magnetic resonance (NMR) models, enabling immersive visual comparison of the local conformation with the local NMR experimental data, including target curves for residual dipolar couplings (RDCs). CONCLUSION: The promise of KinImmerse for production-level molecular research in the DiVE is shown by the locally co-centered RDC visualization developed there, which gave new insights now being pursued in wider data analysis.