Quantifying time perception during virtual reality gameplay using a multimodal biosensor-instrumented headset: a feasibility study.

We have all experienced the sense of time slowing down when we are bored or speeding up when we are focused, engaged, or excited about a task. In virtual reality (VR), perception of time can be a key aspect related to flow, immersion, engagement, and ultimately, to overall quality of experience. While several studies have explored changes in time perception using questionnaires, limited studies have attempted to characterize them objectively. In this paper, we propose the use of a multimodal biosensor-embedded VR headset capable of measuring electroencephalography (EEG), electrooculography (EOG), electrocardiography (ECG), and head movement data while the user is immersed in a virtual environment. Eight gamers were recruited to play a commercial action game comprised of puzzle-solving tasks and first-person shooting and combat. After gameplay, ratings were given across multiple dimensions, including (1) the perception of time flowing differently than usual and (2) the gamers losing sense of time. Several features were extracted from the biosignals, ranked based on a two-step feature selection procedure, and then mapped to a predicted time perception rating using a Gaussian process regressor. Top features were found to come from the four signal modalities and the two regressors, one for each time perception scale, were shown to achieve results significantly better than chance. An in-depth analysis of the top features is presented with the hope that the insights can be used to inform the design of more engaging and immersive VR experiences.

Immersive media experience: a survey of existing methods and tools for human influential factors assessment.

Virtual reality (VR) applications, especially those where the user is untethered to a computer, are becoming more prevalent as new hardware is developed, computational power and artificial intelligence algorithms are available, and wireless communication networks are becoming more reliable, fast, and providing higher reliability. In fact, recent projections show that by 2022 the number of VR users will double, suggesting the sector was not negatively affected by the worldwide COVID-19 pandemic. The success of any immersive communication system is heavily dependent on the user experience it delivers, thus now more than ever has it become crucial to develop reliable models of immersive media experience (IMEx). In this paper, we survey the literature for existing methods and tools to assess human influential factors (HIFs) related to IMEx. In particular, subjective, behavioural, and psycho-physiological methods are covered. We describe tools available to monitor these HIFs, including the user's sense of presence and immersion, cybersickness, and mental/affective states, as well as their role in overall experience. Special focus is placed on psycho-physiological methods, as it was found that such in-depth evaluation was lacking from the existing literature. We conclude by touching on emerging applications involving multiple-sensorial immersive media and provide suggestions for future research directions to fill existing gaps. It is hoped that this survey will be useful for researchers interested in building new immersive (adaptive) applications that maximize user experience.

A frequency-band coupling model of EEG signals can capture features from an input audio stimulus.

Despite decades of research, the features of an input audio stimulus that are encoded in an electroencephalogram (EEG) are still not clearly identified. We wish to investigate whether a frequency-band coupling model that estimates the cortical neural activity from EEGs can capture the important features of an input audio stimulus. To do so, EEG recordings were acquired from 8 subjects during a listening task where the vowels a, i and u were randomly presented. The neural activity was estimated from the EEG using a frequency-band coupling model that combined the EEG's phase in the delta band (2 Hz-4 Hz) and its amplitude in the gamma band (30 Hz-100 Hz). To investigate if the estimated neural activity could capture relevant features of an input audio stimulus, we fitted a generalized linear model (GLM) to the estimated neural activity and applied a statistical relative deviance metric to evaluate how important is the input audio stimulus in the estimated neural activity. We demonstrate that the input audio stimulus is the main component explaining the estimated neural activity and that other aspects such as the contribution of the surrounding network dynamics do not contribute significantly to the estimated neural activity. These results confirm that the features of the EEG used in the coupling model, namely the phase of the delta band and the power of the gamma band, do encode relevant aspects of an input audio signal. This non-invasive approach could be used, for example, to study how the presence of spectro-temporal features in the estimated neural activity is modified depending on different listening conditions or types of input sounds.