Fast & scrupulous: Gesture-based alarms improve accuracy and reaction times under various mental workload levels. An ERSP study.

In high-risk environments, fast and accurate responses to warning systems are essential to efficiently handle emergency situations. The aim of the present study was twofold: 1) investigating whether hand action videos (i.e., gesture alarms) trigger faster and more accurate responses than text alarm messages (i.e., written alarms), especially when mental workload (MWL) is high; and 2) investigating the brain activity in response to both types of alarms as a function of MWL. Regardless of MWL, participants (N = 28) were found to be both faster and more accurate when responding to gesture alarms than to written alarms. Brain electrophysiological results suggest that this greater efficiency might be due to a facilitation of the action execution, reflected by the decrease in mu and beta power observed around the response time window observed at C3 and C4 electrodes. These results suggest that gesture alarms may improve operators' performances in emergency situations.

Investigating the impact of offer frame manipulations on responders playing the ultimatum game.

The present study was designed to test the impact of frame manipulations on the decision-making of responders playing the ultimatum game. Experiment 1 investigated responders' event-related potentials (ERPs) measured in response to the offers as a function of the frame (i.e., negative: "the proposer keeps" versus positive: "the proposer offers"). While no difference in acceptation rate was found as a function of the offer's frame, electrophysiological results suggest that the stronger negative affective response to the offers in the negative frame (N400) was successfully reappraised by the responders (P600), possibly explaining why the offer frame manipulation did not modulate acceptation rates. No framing effect was found when the ultimatum game was played in its one-shot version (Experiment 2), suggesting that repeated measurements did not affect responders' behavior. However, an offer framing effect was found in female (but not in male) responders, when the complexity of the game statement increased, presumably recruiting more cognitive resources and taxing the reappraisal process (Experiment 3). Taken together, these results suggest that framing manipulations are associated with complex affective and cognitive processes, supporting the cognitive-affective tradeoff model.

Busy and confused? High risk of missed alerts in the cockpit: An electrophysiological study.

The ability to react to unexpected auditory stimuli is critical in complex settings such as aircraft cockpits or air traffic control towers, characterized by high mental load and highly complex auditory environments (i.e., many different auditory alerts). Evidence shows that both factors can negatively impact auditory attention and prevent appropriate reactions. In the present study, 60 participants performed a simulated aviation task varying in terms of mental load (no, low, high) concurrently to a tone detection paradigm in which the complexity of the auditory environment (i.e., auditory load) was manipulated (1, 2 or 3 different tones). We measured both detection performance (miss, false alarm, d') and brain activity (event-related potentials) associated with the target tone. Our results showed that both mental and auditory loads affected target tone detection performance. Importantly, their combined effects had a large impact on the percentage of missed target tones. While, in the no mental load condition, miss rate was very low with 1 (0.53%) and 2 tones (1.11%), it increased drastically with 3 tones (24.44%), and this effect was accentuated as mental load increased, yielding to the higher miss rate in the 3-tone paradigm under high mental load conditions (68.64%). Increased mental and auditory loads and miss rates were associated with disrupted brain responses to the target tone, as shown by a reduced P3b amplitude. In sum, our results highlight the importance of balancing mental and auditory loads to maintain efficient reactions to alarms in complex working environment.

Facing successfully high mental workload and stressors: An fMRI study.

The present fMRI study aimed at highlighting patterns of brain activations and autonomic activity when confronted with high mental workload and the threat of auditory stressors. Twenty participants performed a complex cognitive task in either safe or aversive conditions. Our results showed that increased mental workload induced recruitment of the lateral frontoparietal executive control network (ECN), along with disengagement of medial prefrontal and posterior cingulate regions of the default mode network (DMN). Mental workload also elicited an increase in heart rate and pupil diameter. Task performance did not decrease under the threat of stressors, most likely due to efficient inhibition of auditory regions, as reflected by a large decrement of activity in the superior temporal gyri. The threat of stressors was also accompanied with deactivations of limbic regions of the salience network (SN), possibly reflecting emotional regulation mechanisms through control from dorsal medial prefrontal and parietal regions, as indicated by functional connectivity analyses. Meanwhile, the threat of stressors induced enhanced ECN activity, likely for improved attentional and cognitive processes toward the task, as suggested by increased lateral prefrontal and parietal activations. These fMRI results suggest that measuring the balance between ECN, SN, and DMN recruitment could be used for objective mental state assessment. In this sense, an extra recruitment of task-related regions and a high ratio of lateral versus medial prefrontal activity may represent a relevant marker of increased but efficient mental effort, while the opposite may indicate a disengagement from the task due to mental overload and/or stressors.

Visual scanning strategies in the cockpit are modulated by pilots' expertise: A flight simulator study.

During a flight, pilots must rigorously monitor their flight instruments since it is one of the critical activities that contribute to update their situation awareness. The monitoring is cognitively demanding, but is necessary for timely intervention in the event of a parameter deviation. Many studies have shown that a large part of commercial aviation accidents involved poor cockpit monitoring from the crew. Research in eye-tracking has developed numerous metrics to examine visual strategies in fields such as art viewing, sports, chess, reading, aviation, and space. In this article, we propose to use both basic and advanced eye metrics to study visual information acquisition, gaze dispersion, and gaze patterning among novices and pilots. The experiment involved a group of sixteen certified professional pilots and a group of sixteen novice during a manual landing task scenario performed in a flight simulator. The two groups landed three times with different levels of difficulty (manipulated via a double task paradigm). Compared to novices, professional pilots had a higher perceptual efficiency (more numerous and shorter dwells), a better distribution of attention, an ambient mode of visual attention, and more complex and elaborate visual scanning patterns. We classified pilot's profiles (novices-experts) by machine learning based on Cosine KNN (K-Nearest Neighbors) using transition matrices. Several eye metrics were also sensitive to the landing difficulty. Our results can benefit the aviation domain by helping to assess the monitoring performance of the crews, improve initial and recurrent training and ultimately reduce incidents, and accidents due to human error.

Human Mirror Neuron System Based Alarms in the Cockpit: A Neuroergonomic Evaluation.

Controlled Flight Into Terrain (CFIT) events still remain among the deadliest accidents in aviation. When facing the possible occurrence of such an event, pilots have to immediately react to the ground proximity alarm ("Pull Up" alarm) in order to avoid the impending collision. However, the pilots' reaction to this alarm is not always optimal. This may be at least partly due to the low visual saliency of the current alarm and the deleterious effects of stress that alleviate the pilot's reactions. In the present study, two experiments (in a laboratory and in a flight simulator) were conducted to (1) investigate whether hand gesture videos (a hand pulling back the sidestick) can trigger brainwave frequencies related to the mirror neuron system; (2) determine whether enhancing the visual characteristics of the "Pull Up" alarm could improve pilots' response times. Electrophysiological results suggest that hand gesture videos attracted more participants' attention (greater alpha desynchronization in the parieto-occipital area) and possibly triggered greater activity of the mirror neuron system (greater mu and beta desynchronizations at central electrodes). Results obtained in the flight simulator revealed that enhancing the visual characteristics of the original "Pull Up" alarm improved the pilots' reaction times. However, no significant difference in reaction times between an enlarged "Pull Up" inscription and the hand gesture video was found. Further work is needed to determine whether mirror neuron system based alarms could bring benefits for flight safety, in particular, these alarms should be assessed during a high stress context.

The diagnosticity of psychophysiological signatures: Can we disentangle mental workload from acute stress with ECG and fNIRS?

The ability to identify reliable and sensitive physiological signatures of psychological dimensions is key to developing intelligent adaptive systems that may in turn help to mitigate human error in complex operations. The challenge of this endeavor lies with diagnosticity. Despite different underlying causes, the physiological correlates of workload and acute psychological stress manifest in rather similar ways and can be easily confounded. The current work aimed to build a diagnostic model of mental state through the simultaneous classification of mental workload (varied through three levels of the n-back task) and acute stress (the presence/absence of aversive sounds) with machine learning. Using functional near infrared spectroscopy (fNIRS) and electrocardiography (ECG), the model's classifiers was above-chance to disentangle variations of mental workload from variations of acute stress. Both ECG and fNIRS could predict mental workload level, the best accuracy resulted from the two measures in combination. Stress level could not be accurately diagnosed through ECG alone, only with fNIRS or ECG and fNIRS combined. Individual calibration may be important since stress classification was more accurate for those with higher subjective state anxiety, perhaps due to a greater sensitivity to stress. Mental workload and stress were both better classified with activity in lateral prefrontal regions of the cortex than the medial areas, and the HbO2 signal generally lead to better classification than HHB. The current model represents a step forward to finely discriminate different mental states despite their rather analog physiological correlates.

Influences of age, mental workload, and flight experience on cognitive performance and prefrontal activity in private pilots: a fNIRS study.

The effects of aging on cognitive performance must be better understood, especially to protect older individuals who are engaged in risky activities (e.g. aviation). Current literature on executive functions suggests that brain compensatory mechanisms may counter cognitive deterioration due to aging, at least up to certain task load levels. The present study assesses this hypothesis in private pilots engaged in two executive tasks from the standardized CANTAB battery, namely Spatial Working Memory (SWM) and  One Touch Stockings of Cambridge (OTS). Sixty-one pilots from three age groups (young, middle-aged, older) performed these two tasks from low to very high difficulty levels, beyond those reported in previous aging studies. A fNIRS headband measured changes in oxyhemoglobin (HbO2) in the prefrontal cortex. Results confirmed an overall effect of the difficulty level in the three age groups, with a decline in task performance and an increase in prefrontal HbO2 signal. Performance of older relative to younger pilots was impaired in both tasks, with the greatest impairment observed for the highest-load Spatial Working Memory task. Consistent with this behavioral deficit in older pilots, a plateau of prefrontal activity was observed at this highest-load level, suggesting that a ceiling in neural resources was reached. When behavioral performance was either equivalent between age groups or only slightly impaired in the older group, there were not any age-related differences in prefrontal activity. Finally, older pilots with extensive flying experience tend to show better preserved spatial working memory performance when compared to mildly-experienced of the same age group. The present findings are discussed in the frames of HAROLD and CRUNCH theoretical models of cognitive and neural aging, evoking the possibility that piloting expertise may contribute to preserve executive functions throughout adulthood.

Momentary lapse of control: A cognitive continuum approach to understanding and mitigating perseveration in human error.

Everyday complex and stressful real-life situations can overwhelm the human brain to an extent that the person is no longer able to accurately evaluate the situation and persists in irrational actions or strategies. Safety analyses reveal that such perseverative behavior is exhibited by operators in many critical domains, which can lead to potentially fatal incidents. There are neuroimaging evidences of changes in healthy brain functioning when engaged in non-adaptive behaviors that are akin to executive deficits such as perseveration shown in patients with brain lesion. In this respect, we suggest a cognitive continuum whereby stressors can render the healthy brain temporarily impaired. We show that the dorsolateral prefrontal cortex is a key structure for executive and attentional control whereby any transient (stressors, neurostimulation) or permanent (lesion) impairment compromises adaptive behavior. Using this neuropsychological insight, we discuss solutions involving training, neurostimulation, and the design of cognitive countermeasures for mitigating perseveration.

Author Correction: Mental workload and neural efficiency quantified in the prefrontal cortex using fNIRS.

A correction to this article has been published and is linked from the HTML and PDF versions of this paper. The error has not been fixed in the paper.

Cardiovascular correlates of emotional state, cognitive workload and time-on-task effect during a realistic flight simulation.

In aviation, emotion and cognitive workload can considerably increase the probability of human error. An accurate online physiological monitoring of pilot's mental state could prevent accidents. The heart rate (HR) and heart rate variability (HRV) of 21 private pilots were analysed during two realistic flight simulator scenarios. Emotion was manipulated by a social stressor and cognitive workload with the difficulty of a secondary task. Our results confirmed the sensitivity of the HR to cognitive demand and training effects, with increased HR when the task was more difficult and decreased HR with training (time-on-task). Training was also associated with an increased HRV, with increased values along the flight scenario time course. Finally, the social stressor seemed to provoke an emotional reaction that enhanced motivation and performance on the secondary task. However, this was not reflected by the cardiovascular activity.

Mental workload and neural efficiency quantified in the prefrontal cortex using fNIRS.

An improved understanding of how the brain allocates mental resources as a function of task difficulty is critical for enhancing human performance. Functional near infrared spectroscopy (fNIRS) is a field-deployable optical brain monitoring technology that provides a direct measure of cerebral blood flow in response to cognitive activity. We found that fNIRS was sensitive to variations in task difficulty in both real-life (flight simulator) and laboratory settings (tests measuring executive functions), showing increased concentration of oxygenated hemoglobin (HbO2) and decreased concentration of deoxygenated hemoglobin (HHb) in the prefrontal cortex as the tasks became more complex. Intensity of prefrontal activation (HbO2 concentration) was not clearly correlated to task performance. Rather, activation intensity shed insight on the level of mental effort, i.e., how hard an individual was working to accomplish a task. When combined with performance, fNIRS provided an estimate of the participants' neural efficiency, and this efficiency was consistent across levels of difficulty of the same task. Overall, our data support the suitability of fNIRS to assess the mental effort related to human operations and represents a promising tool for the measurement of neural efficiency in other contexts such as training programs or the clinical setting.

Can an aircraft be piloted via sonification with an acceptable attentional cost? A comparison of blind and sighted pilots.

In the aeronautics field, some authors have suggested that an aircraft's attitude sonification could be used by pilots to cope with spatial disorientation situations. Such a system is currently used by blind pilots to control the attitude of their aircraft. However, given the suspected higher auditory attentional capacities of blind people, the possibility for sighted individuals to use this system remains an open question. For example, its introduction may overload the auditory channel, which may in turn alter the responsiveness of pilots to infrequent but critical auditory warnings. In this study, two groups of pilots (blind versus sighted) performed a simulated flight experiment consisting of successive aircraft maneuvers, on the sole basis of an aircraft sonification. Maneuver difficulty was varied while we assessed flight performance along with subjective and electroencephalographic (EEG) measures of workload. The results showed that both groups of participants reached target-attitudes with a good accuracy. However, more complex maneuvers increased subjective workload and impaired brain responsiveness toward unexpected auditory stimuli as demonstrated by lower N1 and P3 amplitudes. Despite that the EEG signal showed a clear reorganization of the brain in the blind participants (higher alpha power), the brain responsiveness to unexpected auditory stimuli was not significantly different between the two groups. The results suggest that an auditory display might provide useful additional information to spatially disoriented pilots with normal vision. However, its use should be restricted to critical situations and simple recovery or guidance maneuvers.

Neural and psychophysiological correlates of human performance under stress and high mental workload.

In our anxiogenic and stressful world, the maintenance of an optimal cognitive performance is a constant challenge. It is particularly true in complex working environments (e.g. flight deck, air traffic control tower), where individuals have sometimes to cope with a high mental workload and stressful situations. Several models (i.e. processing efficiency theory, cognitive-energetical framework) have attempted to provide a conceptual basis on how human performance is modulated by high workload and stress/anxiety. These models predict that stress can reduce human cognitive efficiency, even in the absence of a visible impact on the task performance. Performance may be protected under stress thanks to compensatory effort, but only at the expense of a cognitive cost. Yet, the psychophysiological cost of this regulation remains unclear. We designed two experiments involving pupil diameter, cardiovascular and prefrontal oxygenation measurements. Participants performed the Toulouse N-back Task that intensively engaged both working memory and mental calculation processes under the threat (or not) of unpredictable aversive sounds. The results revealed that higher task difficulty (higher n level) degraded the performance and induced an increased tonic pupil diameter, heart rate and activity in the lateral prefrontal cortex, and a decreased phasic pupil response and heart rate variability. Importantly, the condition of stress did not impact the performance, but at the expense of a psychophysiological cost as demonstrated by lower phasic pupil response, and greater heart rate and prefrontal activity. Prefrontal cortex seems to be a central region for mitigating the influence of stress because it subserves crucial functions (e.g. inhibition, working memory) that can promote the engagement of coping strategies. Overall, findings confirmed the psychophysiological cost of both mental effort and stress. Stress likely triggered increased motivation and the recruitment of additional cognitive resources that minimize its aversive effects on task performance (effectiveness), but these compensatory efforts consumed resources that caused a loss of cognitive efficiency (ratio between performance effectiveness and mental effort).

The Role of Cognitive and Perceptual Loads in Inattentional Deafness.

The current study examines the role of cognitive and perceptual loads in inattentional deafness (the failure to perceive an auditory stimulus) and the possibility to predict this phenomenon with ocular measurements. Twenty participants performed Air Traffic Control (ATC) scenarios-in the Laby ATC-like microworld-guiding one (low cognitive load) or two (high cognitive load) aircraft while responding to visual notifications related to 7 (low perceptual load) or 21 (high perceptual load) peripheral aircraft. At the same time, participants were played standard tones which they had to ignore (probability = 0.80), or deviant tones (probability = 0.20) which they had to report. Behavioral results showed that 28.76% of alarms were not reported in the low cognitive load condition and up to 46.21% in the high cognitive load condition. On the contrary, perceptual load had no impact on the inattentional deafness rate. Finally, the mean pupil diameter of the fixations that preceded the target tones was significantly lower in the trials in which the participants did not report the tones, likely showing a momentary lapse of sustained attention, which in turn was associated to the occurrence of inattentional deafness.

High Working Memory Load Impairs Language Processing during a Simulated Piloting Task: An ERP and Pupillometry Study.

Given the important amount of visual and auditory linguistic information that pilots have to process, operating an aircraft generates a high working-memory load (WML). In this context, the ability to focus attention on relevant information and to remain responsive to concurrent stimuli might be altered. Consequently, understanding the effects of WML on the processing of both linguistic targets and distractors is of particular interest in the study of pilot performance. In the present work, participants performed a simplified piloting task in which they had to follow one of three colored aircraft, according to specific written instructions (i.e., the written word for the color corresponding to the color of one of the aircraft) and to ignore either congruent or incongruent concurrent auditory distractors (i.e., a spoken name of color). The WML was manipulated with an n-back sub-task. Participants were instructed to apply the current written instruction in the low WML condition, and the 2-back written instruction in the high WML condition. Electrophysiological results revealed a major effect of WML at behavioral (i.e., decline of piloting performance), electrophysiological, and autonomic levels (i.e., greater pupil diameter). Increased WML consumed resources that could not be allocated to the processing of the linguistic stimuli, as indexed by lower P300/P600 amplitudes. Also, significantly, lower P600 responses were measured in incongruent vs. congruent trials in the low WML condition, showing a higher difficulty reorienting attention toward the written instruction, but this effect was canceled in the high WML condition. This suppression of interference in the high load condition is in line with the engagement/distraction trade-off model. We propose that P300/P600 components could be reliable indicators of WML and that they allow an estimation of its impact on the processing of linguistic stimuli.

The Responders' Gender Stereotypes Modulate the Strategic Decision-Making of Proposers Playing the Ultimatum Game.

Despite the wealth of studies investigating factors affecting decisions, not much is known about the impact of stereotypical beliefs on strategic economic decision-making. In the present study, we used the ultimatum game paradigm to investigate how participants playing as proposer modulate their strategic economic behavior, according to their game counterparts' stereotypical identity (i.e., responders). The latter were introduced to the participants using occupational role nouns stereotypically marked with gender paired with feminine or masculine proper names (e.g., linguist-Anna; economist-David; economist-Cristina; linguist-Leonardo). When playing with male-stereotyped responders, proposers quickly applied the equity rule, behaving fairly, while they adopted a strategic behavior with responders characterized by female stereotypes. They were also longer to make their offers to female than to male responders but both kinds of responders received comparable offers, suggesting a greater cognitive effort to treat females as equally as males. The present study explicitly demonstrates that gender stereotypical information affect strategic economic decision-making and highlights a possible evolution of gender discrimination into a more insidious discrimination toward individuals with female characteristics.

N270 sensitivity to conflict strength and working memory: A combined ERP and sLORETA study.

The event-related potential N270 component is known to be an electrophysiological marker of the supramodal conflict processing. However little is know about the factors that may modulate its amplitude. In particular, among all studies that have investigated the N270, little or no control of the conflict strength and of the load in working memory have been done leaving a lack in the understanding of this component. We designed a spatial audiovisual conflict task with simultaneous target and cross-modal distractor to evaluate the N270 sensitivity to the conflict strength (i.e., visual target with auditory distractor or auditory target with visual distractor) and the load in working memory (goal task maintenance with frequent change in the target modality). In a first session, participants had to focus on one modality for the target position to be considered (left-hand or right-hand) while the distractor could be at the same side (compatible) or at opposite side (incompatible). In a second session, we used the same set of stimuli as in the first session with an additional distinct auditory signal that clued the participants to frequently switch between the auditory and the visual targets. We found that (1) reaction times and N270 amplitudes for conflicting situations were larger within the auditory target condition compared to the visual one, (2) the increase in target maintenance effort led to equivalent increase of both reaction times and N270 amplitudes within all conditions and (3) the right dorsolateral prefrontal cortex current density was higher for both conflicting and active maintenance of the target situations. These results provide new evidence that the N270 component is an electrophysiological marker of the supramodal conflict processing that is sensitive to the conflict strength and that conflict processing and active maintenance of the task goal are two functions of a common executive attention system.

Sex and the money--How gender stereotypes modulate economic decision-making: An ERP study.

In the present event-related potential study, we investigated whether and how participants playing the ultimatum game as responders modulate their decisions according to the proposers' stereotypical identity. The proposers' identity was manipulated using occupational role nouns stereotypically marked with gender (e.g., Teacher; Engineer), paired with either feminine or masculine proper names (e.g., Anna; David). Greater FRN amplitudes reflected the early processing of the conflict between the strategic rule (i.e., earning as much money as possible) and ready-to-go responses (i.e., refusing unequal offers and discriminating proposers according to their stereotype). Responders were found to rely on a dual-process system (i.e., automatic and heuristic-based system 1 vs. cognitively costly and deliberative system 2), the P300 amplitude reflecting the switch from a decision making system to another. Greater P300 amplitudes were found in response to both fair and unfair offers and male-stereotyped proposers' offers reflecting an automatic decision making based on heuristics, while lower P300 amplitudes were found in response to 3€ offers and the female-stereotyped proposers' offers reflecting a more deliberative reasoning. Overall, the results indicate that participants were more motivated to engage in a costly deliberative reasoning associated with an increase in acceptation rate when playing with female-stereotyped proposers, who may have induced more positive and emphatic feelings in the participants than did male-stereotyped proposers. Then, we assume that people with an occupation stereotypically marked with female gender and engaged in an economic negotiation may benefit from their occupation at least in the case their counterparts lose their money if the negotiation fails.