AudioGest: Gesture-based Interaction for Virtual Reality using Audio Devices.

Current virtual reality (VR) system takes gesture interaction based on camera, handle and touch screen as one of the mainstream interaction methods, which can provide accurate gesture input for it. However, limited by application forms and the volume of devices, these methods cannot extend the interaction area to such surfaces as walls and tables. To address the above challenge, we propose AudioGest, a portable, plug-and-play system that detects the audio signal generated by finger tapping and sliding on the surface through a set of microphone devices without extensive calibration. First, an audio synthesis-recognition pipeline based on micro-contact dynamics simulation is constructed to generate modal audio synthesis from different materials and physical properties. Then the accuracy and effectiveness of the synthetic audio are verified by mixing the synthetic audio with real audio proportionally as the training sets. Finally, a series of desktop office applications are developed to demonstrate the application potential of AudioGest's scalability and versatility in VR scenarios.

Exploring Bimanual Haptic Feedback for Spatial Search in Virtual Reality.

Spatial search tasks are common and crucial in many Virtual Reality (VR) applications. Traditional methods to enhance the performance of spatial search often employ sensory cues such as visual, auditory, or haptic feedback. However, the design and use of bimanual haptic feedback with two VR controllers for spatial search in VR remains largely unexplored. In this work, we explored bimanual haptic feedback with various combinations of haptic properties, where four types of bimanual haptic feedback were designed, for spatial search tasks in VR. Two experiments were designed to evaluate the effectiveness of bimanual haptic feedback on spatial direction guidance and search in VR. The results from the first experiment reveal that our proposed bimanual haptic schemes significantly enhanced the recognition of spatial directions in terms of accuracy and speed compared to spatial audio feedback. The second experiment's findings suggest that the performance of bimanual haptic feedback was comparable to or even better than the visual arrow, especially in reducing the angle of head movement and enhancing searching targets behind the participants, which was supported by subjective feedback as well. Based on these findings, we have derived a set of design recommendations for spatial search using bimanual haptic feedback in VR.

Effects of Transfer Functions and Body Parts on Body-Centric Locomotion in Virtual Reality.

Body-centric locomotion allows users to control both movement speed and direction with body parts (e.g., head tilt, arm swing or torso lean) to navigate in virtual reality (VR). However, there is little research to systematically investigate the effects of body parts for speed and direction control on virtual locomotion by taking in account different transfer functions(L: linear function, P: power function, and CL: piecewise function with constant and linear function). Therefore, we conducted an experiment to evaluate the combinational effects of the three factors (body parts for direction control, body parts for speed control, and transfer functions) on virtual locomotion. Results showed that (1) the head outperformed the torso for movement direction control in task completion time and environmental collisions; (2) Arm-based speed control led to shorter traveled distances than both head and knee. Head-based speed control had fewer environmental collisions than knee; (3) Body-centric locomotion with CL function was faster but less accurate than both L and P functions. Task time significantly decreased from P, L to CL functions, while traveled distance and overshoot significantly increased from P, L to CL functions. L function was rated with the highest score of USE-S, -pragmatic and -hedonic; (4) Transfer function had a significant main effect on motion sickness: the participants felt more headache and nausea when performing locomotion with CL function. Our results provide implications for body-centric locomotion design in VR applications.

Alpha Frequency Intervention by Electrical Stimulation to Improve Performance in Mu-Based BCI.

The accuracy of brain-computer interfaces (BCIs) is important for effective communication and control. The mu-based BCI is one of the most widely used systems, of which the related methods to improve users' accuracy are still poorly studied, especially for the BCI illiteracy. Here, we examined a way to enhance mu-based BCI performance by electrically stimulating the ulnar nerve of the contralateral wrist at the alpha frequency (10 Hz) during left- and right-hand motor imagination in two BCI groups (literate and illiterate). We demonstrate that this alpha frequency intervention enhances the classification accuracy between left- and right-hand motor imagery from 66.41% to 81.57% immediately after intervention and to 75.28% two days after intervention in the BCI illiteracy group, while classification accuracy improves from 82.12% to 91.84% immediately after intervention and to 89.03% two days after intervention in the BCI literacy group. However, the classification accuracy did not change before and after the sham intervention (no electrical stimulation). Furthermore, the ERD on the primary sensorimotor cortex during left- or right-hand motor imagery tasks was more visible at the mu-rhythm (8-13 Hz) after alpha frequency intervention. Alpha frequency intervention increases the mu-rhythm power difference between left- and right-hand motor imagery tasks. These results provide evidence that alpha frequency intervention is an effective way to improve BCI performance by regulating the mu-rhythm which might provide a way to reduce BCI illiteracy.

Enhancing Mu-based BCI Performance with Rhythmic Electrical Stimulation at Alpha Frequency.

The accuracy of brain-computer interfaces (BCIs) is important for effective communication and control. The mu-based BCI is one of the widely used systems, of which the related methods to improve users' accuracy is still poorly studied. Here, we examined the way to enhance the mu-based BCI performance by rhythmic electrical stimulation on the ulnar nerve at the contralateral wrist at the alpha frequency (10 Hz) during the left-and right-hand motor imagery. Time-frequency analysis, spectral analysis, and discriminant analysis were performed on the electroencephalograph (EEG) data before and after the intervention of electrical stimulation in 9 healthy subjects. We found that the ERD/S on the somatosensory and motor cortex during left-or right-hand imagination was more obvious at the mu rhythm after intervention. Furthermore, average classification accuracy between left-and right-hand imagery significantly increased from 78.43% to 88.17% after intervention, suggesting that the electrical stimulation at alpha frequency effectively regulates the brain's mu rhythm and enhances the discriminability of the left-hand and right-hand imagination tasks. These results provide evidence that the electrical stimulation at the alpha frequency is an effective way to improve the mu-based BCI performance.