Multi-Level Precues for Guiding Tasks Within and Between Workspaces in Spatial Augmented Reality.

We explore Spatial Augmented Reality (SAR) precues (predictive cues) for procedural tasks within and between workspaces and for visualizing multiple upcoming steps in advance. We designed precues based on several factors: cue type, color transparency, and multi-level (number of precues). Precues were evaluated in a procedural task requiring the user to press buttons in three surrounding workspaces. Participants performed fastest in conditions where tasks were linked with line cues with different levels of color transparency. Precue performance was also affected by whether the next task was in the same workspace or a different one.

Adaptive Reset Techniques for Haptic Retargeted Interaction.

This article presents a set of adaptive reset techniques for use with haptic retargeting systems focusing on interaction with hybrid virtual reality interfaces that align with a physical interface. Haptic retargeting between changing physical and virtual targets requires a reset where the physical and virtual hand positions are re-aligned. We present a modified Point technique to guide the user in the direction of their next interaction such that the remaining distance to the target is minimized upon completion of the reset. This, along with techniques drawn from existing work are further modified to consider the angular and translational gain of each redirection and identify the optimal position for the reset to take place. When the angular and translational gain is within an acceptable range, the reset can be entirely omitted. This enables continuous retargeting between targets removing interruptions from a sequence of retargeted interactions. These techniques were evaluated in a user study which showed that adaptive reset techniques can provide a significant decrease in task completion time, travel distance, and the number of user errors.

Event Related Brain Responses Reveal the Impact of Spatial Augmented Reality Predictive Cues on Mental Effort.

This article presents the results from a Spatial Augmented Reality (SAR) study which evaluated the cognitive cost of several predictive cues. Participants performed a validated procedural button pressing task, where the predictive cue annotations guided them to the upcoming task. While existing research has evaluated predictive cues based on their performance and self-rated mental effort, actual cognitive cost has yet to be investigated. To measure the user's brain activity, this study utilized electroencephalogram (EEG) recordings. Cognitive load was evaluated by measuring brain responses for a secondary auditory oddball task, with reduced brain responses to oddball tones expected when cognitive load in the primary task is highest. A simple monitor n-back task and procedural task comparing monitor versus SAR were conducted, followed by a version of the procedural task comparing the SAR predictive cues. Results from the brain responses were able to distinguish between performance enhancing cues with a high and low cognitive load. Electrical brain responses also revealed that having an arc or arrow guide towards the upcoming task required the least amount of mental effort.

There Is No Spoon: Evaluating Performance, Space Use, and Presence with Expert Domain Users in Immersive Analytics.

Immersive analytics turns the very space surrounding the user into a canvas for data analysis, supporting human cognitive abilities in myriad ways. We present the results of a design study, contextual inquiry, and longitudinal evaluation involving professional economists using a Virtual Reality (VR) system for multidimensional visualization to explore actual economic data. Results from our preregistered evaluation highlight the varied use of space depending on context (exploration vs. presentation), the organization of space to support work, and the impact of immersion on navigation and orientation in the 3D analysis space.

Conveying spatial awareness cues in xR collaborations.

Spatial Augmented Reality (SAR) systems can be suitably combined with other existing Extended Reality (xR) technologies to support collaboration. In existing strategies, users unencumbered by a viewing technology, such as a tablet interface or a head-mounted display, must rely on the transmission of their collaborators' positioning through interpreting a first-person camera view. This design creates a seam between a user's experience of the augmented physical environment in SAR, and their collaborators' experience inside the virtual environment. To assist in development and evaluation of spatial cues to support spatial awareness in SAR environments, an egocentric spatial-communication taxonomy is presented given two determining dimensions, a cue's attachment (physical/virtual) and animation (local/world). We developed four egocentric cues which characterize the four independent dimensions of the matrix: arrow, path, glow, and radial, and a single exocentric world in miniature visualization. Our study shows that virtual attachment cues are preferred, providing the highest accuracy, highest performance when collaborators are occluded, and produce the least mental effort when used with a single virtual collaborator. For multiple collaborators however, the virtual attached, world animated radial cue produces significant increases in mental load and reductions in preference, demonstrating the impact of visual augmentation clutter. The single exocentric visualization produced higher levels of head movement, and poorer accuracy, however the novelty of the visualization produced positive qualitative results.

Superman vs Giant: A Study on Spatial Perception for a Multi-Scale Mixed Reality Flying Telepresence Interface.

The advancements in Mixed Reality (MR), Unmanned Aerial Vehicle, and multi-scale collaborative virtual environments have led to new interface opportunities for remote collaboration. This paper explores a novel concept of flying telepresence for multi-scale mixed reality remote collaboration. This work could enable remote collaboration at a larger scale such as building construction. We conducted a user study with three experiments. The first experiment compared two interfaces, static and dynamic IPD, on simulator sickness and body size perception. The second experiment tested the user perception of a virtual object size under three levels of IPD and movement gain manipulation with a fixed eye height in a virtual environment having reduced or rich visual cues. Our last experiment investigated the participant's body size perception for two levels of manipulation of the IPDs and heights using stereo video footage to simulate a flying telepresence experience. The studies found that manipulating IPDs and eye height influenced the user's size perception. We present our findings and share the recommendations for designing a multi-scale MR flying telepresence interface.

A Comparison of Predictive Spatial Augmented Reality Cues for Procedural Tasks.

Previous research has demonstrated that Augmented Reality can reduce a user's task response time and mental effort when completing a procedural task. This paper investigates techniques to improve user performance and reduce mental effort by providing projector-based Spatial Augmented Reality predictive cues for future responses. The objective of the two experiments conducted in this study was to isolate the performance and mental effort differences from several different annotation cueing techniques for simple (Experiment 1) and complex (Experiment 2) button-pressing tasks. Comporting with existing cognitive neuroscience literature on prediction, attentional orienting, and interference, we hypothesized that for both simple procedural tasks and complex search-based tasks, having a visual cue guiding to the next task's location would positively impact performance relative to a baseline, no-cue condition. Additionally, we predicted that direction-based cues would provide a more significant positive impact than target-based cues. The results indicated that providing a line to the next task was the most effective technique for improving the users' task time and mental effort in both the simple and complex tasks.

Narrative and Spatial Memory for Jury Viewings in a Reconstructed Virtual Environment.

This paper showcases one way of how virtual reconstruction can be used in a courtroom. The results of a pilot study on narrative and spatial memory are presented in the context of viewing real and virtual copies of a simulated crime scene. Based on current court procedures, three different viewing options were compared: photographs, a real life visit, and a 3D virtual reconstruction of the scene viewed in a Virtual Reality headset. Participants were also given a written narrative that included the spatial locations of stolen goods and were measured on their ability to recall and understand these spatial relationships of those stolen items. The results suggest that Virtual Reality is more reliable for spatial memory compared to photographs and that Virtual Reality provides a compromise for when physical viewing of crime scenes are not possible. We conclude that Virtual Reality is a promising medium for the court.

Cognitive Cost of Using Augmented Reality Displays.

This paper presents the results of two cognitive load studies comparing three augmented reality display technologies: spatial augmented reality, the optical see-through Microsoft HoloLens, and the video see-through Samsung Gear VR. In particular, the two experiments focused on isolating the cognitive load cost of receiving instructions for a button-pressing procedural task. The studies employed a self-assessment cognitive load methodology, as well as an additional dual-task cognitive load methodology. The results showed that spatial augmented reality led to increased performance and reduced cognitive load. Additionally, it was discovered that a limited field of view can introduce increased cognitive load requirements. The findings suggest that some of the inherent restrictions of head-mounted displays materialize as increased user cognitive load.

Immersive Collaborative Analysis of Network Connectivity: CAVE-style or Head-Mounted Display?

High-quality immersive display technologies are becoming mainstream with the release of head-mounted displays (HMDs) such as the Oculus Rift. These devices potentially represent an affordable alternative to the more traditional, centralised CAVE-style immersive environments. One driver for the development of CAVE-style immersive environments has been collaborative sense-making. Despite this, there has been little research on the effectiveness of collaborative visualisation in CAVE-style facilities, especially with respect to abstract data visualisation tasks. Indeed, very few studies have focused on the use of these displays to explore and analyse abstract data such as networks and there have been no formal user studies investigating collaborative visualisation of abstract data in immersive environments. In this paper we present the results of the first such study. It explores the relative merits of HMD and CAVE-style immersive environments for collaborative analysis of network connectivity, a common and important task involving abstract data. We find significant differences between the two conditions in task completion time and the physical movements of the participants within the space: participants using the HMD were faster while the CAVE2 condition introduced an asymmetry in movement between collaborators. Otherwise, affordances for collaborative data analysis offered by the low-cost HMD condition were not found to be different for accuracy and communication with the CAVE2. These results are notable, given that the latest HMDs will soon be accessible (in terms of cost and potentially ubiquity) to a massive audience.

Augmented Reality as a Countermeasure for Sleep Deprivation.

Sleep deprivation is known to have serious deleterious effects on executive functioning and job performance. Augmented reality has an ability to place pertinent information at the fore, guiding visual focus and reducing instructional complexity. This paper presents a study to explore how spatial augmented reality instructions impact procedural task performance on sleep deprived users. The user study was conducted to examine performance on a procedural task at six time points over the course of a night of total sleep deprivation. Tasks were provided either by spatial augmented reality-based projections or on an adjacent monitor. The results indicate that participant errors significantly increased with the monitor condition when sleep deprived. The augmented reality condition exhibited a positive influence with participant errors and completion time having no significant increase when sleep deprived. The results of our study show that spatial augmented reality is an effective sleep deprivation countermeasure under laboratory conditions.

Moment to moment variability in functional brain networks during cognitive activity in EEG data.

Functional brain networks (FBNs) are gaining increasing attention in computational neuroscience due to their ability to reveal dynamic interdependencies between brain regions. The dynamics of such networks during cognitive activity between stimulus and response using multi-channel electroencephalogram (EEG), recorded from 16 healthy human participants are explored in this research. Successive EEG segments of 500[Formula: see text]ms duration starting from the onset of cognitive stimulation have been used to analyze and understand the cognitive dynamics. The approach employs a combination of signal processing techniques, nonlinear statistical measures and graph-theoretical analysis. The efficacy of this approach in detecting and tracking cognitive load induced changes in EEG data is clearly demonstrated using graph metrics. It is revealed that most cognitive activity occurs within approximately 500[Formula: see text]ms of the stimulus presentation in addition to temporal variability in the FBNs. It is shown that mutual information (MI), a nonlinear measure, produces good correlations between the EEG channels thus enabling the construction of FBNs which are sensitive to cognitive load induced changes in EEG. Analyses of the dynamics of FBNs and the visualization approach reveal hard to detect subtle changes in cognitive function and hence may lead to a better understanding of cognitive processing in the brain. The techniques exploited have the potential to detect human cognitive dysfunction (impairments).

Spatial user interfaces for large-scale projector-based augmented reality.

Spatial augmented reality applies the concepts of spatial user interfaces to large-scale, projector-based augmented reality. Such virtual environments have interesting characteristics. They deal with large physical objects, the projection surfaces are nonplanar, the physical objects provide natural passive haptic feedback, and the systems naturally support collaboration between users. The article describes how these features affect the design of spatial user interfaces for these environments and explores promising research directions and application domains.

Geometrically-correct projection-based texture mapping onto a deformable object.

Projection-based Augmented Reality commonly employs a rigid substrate as the projection surface and does not support scenarios where the substrate can be reshaped. This investigation presents a projection-based AR system that supports deformable substrates that can be bent, twisted or folded. We demonstrate a new invisible marker embedded into a deformable substrate and an algorithm that identifies deformations to project geometrically correct textures onto the deformable object. The geometrically correct projection-based texture mapping onto a deformable marker is conducted using the measurement of the 3D shape through the detection of the retro-reflective marker on the surface. In order to achieve accurate texture mapping, we propose a marker pattern that can be partially recognized and can be registered to an object’s surface. The outcome of this work addresses a fundamental vision recognition challenge that allows the underlying material to change shape and be recognized by the system. Our evaluation demonstrated the system achieved geometrically correct projection under extreme deformation conditions. We envisage the techniques presented are useful for domains including prototype development, design, entertainment and information based AR systems.

The efficacy of playing a virtual reality game in modulating pain for children with acute burn injuries: a randomized controlled trial [ISRCTN87413556].

BACKGROUND: The management of burn injuries is reported as painful, distressing and a cause of anxiety in children and their parents. Child's and parents' pain and anxiety, often contributes to extended time required for burns management procedures, in particular the process of changing dressings. The traditional method of pharmacologic analgesia is often insufficient to cover the burnt child's pain, and it can have deleterious side effects 12. Intervention with Virtual Reality (VR) games is based on distraction or interruption in the way current thoughts, including pain, are processed by the brain. Research on adults supports the hypothesis that virtual reality has a positive influence on burns pain modulation. METHODS: This study investigates whether playing a virtual reality game, decreases procedural pain in children aged 5-18 years with acute burn injuries. The paper reports on the findings of a pilot study, a randomised trial, in which seven children acted as their own controls though a series of 11 trials. Outcomes were pain measured using the self-report Faces Scale and findings of interviews with parent/carer and nurses. RESULTS: The average pain scores (from the Faces Scale) for pharmacological analgesia only was, 4.1 (SD 2.9), while VR coupled with pharmacological analgesia, the average pain score was 1.3 (SD 1.8) CONCLUSION: The study provides strong evidence supporting VR based games in providing analgesia with minimal side effects and little impact on the physical hospital environment, as well as its reusability and versatility, suggesting another option in the management of children's acute pain.