ABSTRACT
INTRODUCTION: Despite their popularity in Neuropsychology, reaction time analysis based on the subtraction and additive factors methods is critiqued for not paying adequate attention to the dynamical nature of cognition. Mouse-tracking methods aim to cater to this need by allowing researchers to explore response dynamics during cognitive tasks by recording mouse trajectories. METHODS: A mouse-tracking adaptation of the Simon task is developed to explore decision-making dynamics in different stimulus-response compatibility conditions. The study focuses on the effects of stimulus design decisions on mouse trajectories, including relocation of the choice buttons from the top corners to the bottom and the mid-session reversal of stimulus-response mapping on mouse responses. RESULTS: Consistent with previous studies, significant stimulus-response compatibility effects were observed, where contrasts over mouse-tracking measures had larger effect sizes than simple reaction time contrasts. Moreover, in the conflict trials, asymmetric response trajectories towards the left and right corners were observed. Moving the response buttons from top to bottom increased the degree of asymmetry between the mouse trajectories towards the bottom-left and bottom-right corners during the conflict condition. Finally, in the reverse Simon task, the switch to a new color-response mapping inflicted the largest effect on the average number of y-flips. CONCLUSION: Mouse tracking provides measures suitable for exploring decision-making dynamics beyond classical reaction time analysis, provided asymmetries due to the starting position and response layout are considered during experiment design.
ABSTRACT
Recent advances in neuroimaging technologies have rendered multimodal analysis of operators' cognitive processes in complex task settings and environments increasingly more practical. In this exploratory study, we utilized optical brain imaging and mobile eye tracking technologies to investigate the behavioral and neurophysiological differences among expert and novice operators while they operated a human-machine interface in normal and adverse conditions. In congruence with related work, we observed that experts tended to have lower prefrontal oxygenation and exhibit gaze patterns that are better aligned with the optimal task sequence with shorter fixation durations as compared to novices. These trends reached statistical significance only in the adverse condition where the operators were prompted with an unexpected error message. Comparisons between hemodynamic and gaze measures before and after the error message indicated that experts' neurophysiological response to the error involved a systematic increase in bilateral dorsolateral prefrontal cortex (dlPFC) activity accompanied with an increase in fixation durations, which suggests a shift in their attentional state, possibly from routine process execution to problem detection and resolution. The novices' response was not as strong as that of experts, including a slight increase only in the left dlPFC with a decreasing trend in fixation durations, which is indicative of visual search behavior for possible cues to make sense of the unanticipated situation. A linear discriminant analysis model capitalizing on the covariance structure among hemodynamic and eye movement measures could distinguish experts from novices with 91% accuracy. Despite the small sample size, the performance of the linear discriminant analysis combining eye fixation and dorsolateral oxygenation measures before and after an unexpected event suggests that multimodal approaches may be fruitful for distinguishing novice and expert performance in similar neuroergonomic applications in the field.
