Measuring Engagement of Spectators of Social Digital Games.

The goal of this methodology is to assess explicit and implicit measures of engagement of spectators during social digital games in a group of participants with motion tracking systems. In the context of games that are not confined within a screen, measuring the different dimensions of engagement such as physiological arousal can be challenging. The focus of the study is made on the spectators of the game and the differences in their engagement according to interactivity. Engagement is measured with physiological and self-reported arousal, as well as an engagement questionnaire at the end of the experiment. Physiological arousal is measured with electrodermal activity (EDA) sensors that record the data on a portable device (EDA box). Portability was essential because of the nature of the game, which is akin to a life-size pong and includes many participants that move. To have an overview of the events of the game, three cameras are used to film three angles of the playing field. To synchronize the EDA data with events happening in the game, boxes with digital numbers are used and put in the frames of cameras. Signals are sent from a sync box simultaneously to the EDA boxes and to light boxes. The light boxes show the synchronization numbers to the cameras, and the same numbers are also logged on the EDA data file. That way, it is possible to record EDA of many people that move freely in a large space and synchronize this data with events in the game. In our particular study, we were able to assess the differences in arousal for the different conditions of interactivity. One of the limitations of this method is that the signals cannot be sent farther than 20 meters away. This method is, therefore, appropriate for recording physiological data in games with an unlimited number of players but is restricted to a limited space.

Measuring the Switch Cost of Smartphone Use While Walking.

This paper presents a study protocol to measure the task-switching cost of using a smartphone while walking. This method involves having participants walk on a treadmill under two experimental conditions: a control condition (i.e., simply walking) and a multitasking condition (i.e., texting while walking). During these conditions, the participants must switch between the tasks related to the experimental condition and a direction determining task. This direction task is done with a point-light walker figure, seemingly walking towards the left or the right of the participant. Performance on the direction task represents the participant's task-switching costs. There were two performance measures: 1) correct identification of the direction and 2) response time. EEG data are recorded in order to measure the alpha oscillations and cognitive engagement occurring during the task switch. This method is limited in its ecological validity: pedestrian environments have many stimuli occurring simultaneously and competing for attention. Nonetheless, this method is appropriate for pinpointing task-switching costs. The EEG data allow the study of the underlying mechanisms in the brain that are related to differing task-switching costs. This design allows the comparison between task switching when doing one task at a time, as compared to task switching when multitasking, prior to the stimulus presentation. This allows understanding and pinpointing both the behavioral and neurophysiological impact of these two different task-switching conditions. Furthermore, by correlating the task-switching costs with the brain activity, we can learn more about what causes these behavioral effects. This protocol is an appropriate base for studying the switching cost of different smartphone uses. Different tasks, questionnaires, and other measures can be added to it in order to understand the different factors involved in the task-switching cost of smartphone use while walking.

Texting while walking: An expensive switch cost.

Texting while walking has been highlighted as a dangerous behavior that leads to impaired judgment and accidents. This impairment could be due to task switching which involves activation of the present task and the inhibition of the previous task. However, the relative contributions of these processes and their brain activity have not yet been studied. We addressed this gap by asking participants to discriminate the orientation of an oncoming human shape in a virtual environment while they were: i) walking on a treadmill, or ii) texting while walking on a treadmill. Participants' performance (i.e., correctly identifying if a walker would pass them to their left or right) and electroencephalography (EEG) data was collected. Unsurprisingly, we found that participants performed better while they were only walking than when texting while walking. However, we also found that the diminished performance is differently related to task set inhibition and task set activation in the two conditions. The alpha oscillations, which can be used as an index of task inhibition, have a significantly different relation to performance in the two conditions, the relation being negative when subjects are texting. This may indicate that the more inhibition is needed, the more the performance is affected by texting. To our knowledge, this is the first study to investigate the brain signature of task switching in texting while walking. This finding is the first step in identifying the source of impaired judgment in texting pedestrians and in finding viable solutions to reduce the risks.

Dynamic Threshold Selection for a Biocybernetic Loop in an Adaptive Video Game Context.

Passive Brain-Computer interfaces (pBCIs) are a human-computer communication tool where the computer can detect from neurophysiological signals the current mental or emotional state of the user. The system can then adjust itself to guide the user toward a desired state. One challenge facing developers of pBCIs is that the system's parameters are generally set at the onset of the interaction and remain stable throughout, not adapting to potential changes over time such as fatigue. The goal of this paper is to investigate the improvement of pBCIs with settings adjusted according to the information provided by a second neurophysiological signal. With the use of a second signal, making the system a hybrid pBCI, those parameters can be continuously adjusted with dynamic thresholding to respond to variations such as fatigue or learning. In this experiment, we hypothesize that the adaptive system with dynamic thresholding will improve perceived game experience and objective game performance compared to two other conditions: an adaptive system with single primary signal biocybernetic loop and a control non-adaptive game. A within-subject experiment was conducted with 16 participants using three versions of the game Tetris. Each participant plays 15 min of Tetris under three experimental conditions. The control condition is the traditional game of Tetris with a progressive increase in speed. The second condition is a cognitive load only biocybernetic loop with the parameters presented in Ewing et al. (2016). The third condition is our proposed biocybernetic loop using dynamic threshold selection. Electroencephalography was used as the primary signal and automatic facial expression analysis as the secondary signal. Our results show that, contrary to our expectations, the adaptive systems did not improve the participants' experience as participants had more negative affect from the BCI conditions than in the control condition. We endeavored to develop a system that improved upon the authentic version of the Tetris game, however, our proposed adaptive system neither improved players' perceived experience, nor their objective performance. Nevertheless, this experience can inform developers of hybrid passive BCIs on a novel way to employ various neurophysiological features simultaneously.

Assessing the Multiple Dimensions of Engagement to Characterize Learning: A Neurophysiological Perspective.

In a recent theoretical synthesis on the concept of engagement, Fredricks, Blumenfeld and Paris defined engagement by its multiple dimensions: behavioral, emotional and cognitive. They observed that individual types of engagement had not been studied in conjunction, and little information was available about interactions or synergy between the dimensions; consequently, more studies would contribute to creating finely tuned teaching interventions. Benefiting from the recent technological advances in neurosciences, this paper presents a recently developed methodology to gather and synchronize data on multidimensional engagement during learning tasks. The technique involves the collection of (a) electroencephalography, (b) electrodermal, (c) eye-tracking, and (d) facial emotion recognition data on four different computers. This led to synchronization issues for data collected from multiple sources. Post synchronization in specialized integration software gives researchers a better understanding of the dynamics between the multiple dimensions of engagement. For curriculum developers, these data could provide informed guidelines for achieving better instruction/learning efficiency. This technique also opens up possibilities in the field of brain-computer interactions, where adaptive learning or assessment environments could be developed.