The brain under cognitive workload: Neural networks underlying multitasking performance in the multi-attribute task battery.

Multitasking is a common requirement in many occupations. Considerable research has demonstrated that performance declines as a result of multitasking, and that it engages multiple brain regions. Despite growing evidence suggesting that brain regions operate as networks, minimal research has investigated the cognitive brain networks implicated in multitasking. The Multi-Attribute Task Battery II (MATB) is a common method for assessing multitasking ability that simulates a pilot's operational environment inside an aircraft cockpit. The aim of the present study was to examine multitasking performance on the MATB, and the associated neural patterns underlying performance with functional magnetic resonance imaging (fMRI). Twenty-four participants completed the MATB in the fMRI scanner. Participants completed four runs of the MATB in a 2 (Task: multitasking vs. single tasking) × 2 (Difficulty: hard vs. easy) design. MATB performance was measured as a function of accuracy. We analyzed the fMRI brain scans using both static and dynamic functional connectivity to determine whether there were differences in the connectivity patterns associated with each of the four conditions. A significant interaction between Task and Difficulty was observed such that multitasking performance accuracy, which was derived from the average across tasks, was lower than single tasking in the hard, but not easy, condition. The fMRI data revealed that static and dynamic functional connectivity between the default mode and dorsal attention networks was stronger during multitasking relative to single tasking. The static functional connectivity between the default mode and left frontoparietal networks, along with the dynamic functional connectivity between the dorsal attention and left frontoparietal networks, were both more anti-correlated during multitasking relative to single tasking. Taken together, the static and dynamic functional connectivity analyses provide complementary information to reveal the interactions among cognitive networks that support multitasking performance. Targeting these networks may offer a path to enhance multitasking ability through the application of neurostimulation and neuroenhancement techniques.

Comparing an augmented reality navigation display to an electronic map for military reconnaissance.

The next generation of displays for soldiers may include augmented reality capabilities. One such display, called Mirror in the Sky (MitS), presents survey information in the upper visual field. Using a virtual reality simulation of a military reconnaissance scenario, we compared a MitS prototype to a familiar electronic 2D north-up map. Participants (24 soldiers) were told to follow a prescribed route, detect potential threats, and reroute around them. They also performed a secondary task as a measure of mental workload. At the end of the route, the soldiers were asked to recall the locations of threats and route changes. Participants made better reroute decisions with the north-up map than with MitS, although no differences were observed for threat detection or mental workload. They also scored higher on recall with the north-up map than with MitS. Practitioner Summary: An augmented reality navigation aid was compared to an electronic north-up map in a military reconnaissance scenario, in a virtual reality simulation. Participants made better route decisions and had better recall with the north-up map, but no mental workload differences were found between displays.

A mirror in the sky: the effects of map format and user expertise on navigation performance and mental workload.

A novel map display concept named Mirror in the Sky (MitS) has been introduced to improve performance and reduce workload in navigation tasks. However, this display will be novel to most users and as such, an evaluation of MitS in comparison with more conventional map formats is warranted. This study investigated the effects of map display format (MitS vs. north-up and track-up maps) and user expertise on mental workload (MWL) and performance, using both soldiers (experts) and civilians (novices) as participants. Participants followed a prescribed route to a destination in a virtual environment (route following task) while also performing a secondary task (detection response task). Soldiers generally performed better than civilians. Soldiers reported a higher MWL with MitS than with the north-up map, whereas civilians reported a higher MWL with MitS than with the track-up map. Regardless of user expertise, there were performance and workload challenges with MitS, despite its potential. Practitioner summary: A new map display concept called Mirror in the Sky (MitS) was compared with two conventional map formats: a north-up and track-up map. The experiment tested soldier and civilian users in a route following task. Both groups got further into the route and had fewer obstacle collisions with north-up and track-up maps than they did with the MitS map. MWL measures generally indicated higher workload with MitS. Abbreviations: MitS: mirror in the sky; FFOV: forward field of view; AR: augmented reality; MWL: mental workload; VR: virtual reality; HF: human factors; HR: heart rate; HRV: heart rate variability; DRT: detection response task; DRDC: defence research and development Canada; VE: virtual environment; RT: response time; ANOVA: analysis of variance.

A mirror in the sky: assessment of an augmented reality method for depicting navigational information.

We investigated the efficacy of a novel augmented reality (AR) navigation display called Mirror in the Sky (MitS). AR displays can reduce the distance between virtual imagery content and the user's view of the environment but may have limited benefit for depicting map-based survey information. MitS presents a simulated mirror in the upper visual field, which reflects the topographic layout of the terrain in front of the user. In our experiment, 28 participants used MitS and a track-up Map in virtual reality to perform a route confirmation task, which required participants to decide whether a route could be successfully navigated. A post-trial threat location recall task examined spatial awareness. On that task, accuracy, duration, and subjective workload measures favoured the Map. However, participants with virtual reality experience made more accurate route confirmation decisions with MitS than the Map. Practitioner summary: We compared an augmented reality display called Mirror in the Sky (MitS) to a conventional electronic map for route confirmation and threat location tasks. Although the electronic map showed advantages over MitS on some measures, users with some VR experience performed route confirmation more accurately with MitS than a map.

Cognitive Load and Situation Awareness for Soldiers: Effects of Message Presentation Rate and Sensory Modality.

OBJECTIVE: We sought to determine the influence of message presentation rate (MPR) and sensory modality on soldier cognitive load. BACKGROUND: Soldiers commonly communicate tactical information by radio. The Canadian Army is equipping soldiers with a battle management system (BMS), which also allows them to communicate by text. METHOD: We varied presentation modality (auditory vs. visual) and MPR (fast or slow) in an experiment involving a tactical scenario. Participants (soldiers) received messages and periodically provided situation reports to higher level command, and the scored reports were used to provide a measure of situation awareness (SA). The detection response task (DRT) and NASA-TLX were used to measure cognitive load. RESULTS: The fast MPR reduced DRT accuracy and increased response times relative to slow MPR. The NASA-TLX results also showed higher subjective workload ratings for several subscales with fast MPR. Messages presented visually produced greater cognitive load, with slower DRT response times for the visual than the auditory condition. SA scores were higher with slower MPR and auditory presentation. There was no statistical interaction of presentation modality and rate for any measure. CONCLUSION: Fast MPR and visual presentation increased cognitive load and degraded SA. APPLICATION: These findings show that the DRT can be used to measure workload effectively in a tactical military context and that the method of information presentation affects how soldiers process information in a BMS.

Effects of Resolution, Range, and Image Contrast on Target Acquisition Performance.

OBJECTIVE: We sought to determine the joint influence of resolution, target range, and image contrast on the detection and identification of targets in simulated naturalistic scenes. BACKGROUND: Resolution requirements for target acquisition have been developed based on threshold values obtained using imaging systems, when target range was fixed, and image characteristics were determined by the system. Subsequent work has examined the influence of factors like target range and image contrast on target acquisition. METHOD: We varied the resolution and contrast of static images in two experiments. Participants (soldiers) decided whether a human target was located in the scene (detection task) or whether a target was friendly or hostile (identification task). Target range was also varied (50-400 m). In Experiment 1, 30 participants saw color images with a single target exemplar. In Experiment 2, another 30 participants saw monochrome images containing different target exemplars. RESULTS: The effects of target range and image contrast were qualitatively different above and below 6 pixels per meter of target for both tasks in both experiments. CONCLUSION: Target detection and identification performance were a joint function of image resolution, range, and contrast for both color and monochrome images. APPLICATION: The beneficial effects of increasing resolution for target acquisition performance are greater for closer (larger) targets.

Target Detection and Identification Performance Using an Automatic Target Detection System.

OBJECTIVE: We investigated the effects of automatic target detection (ATD) on the detection and identification performance of soldiers. BACKGROUND: Prior studies have shown that highlighting targets can aid their detection. We provided soldiers with ATD that was more likely to detect one target identity than another, potentially acting as an implicit identification aid. METHOD: Twenty-eight soldiers detected and identified simulated human targets in an immersive virtual environment with and without ATD. Task difficulty was manipulated by varying scene illumination (day, night). The ATD identification bias was also manipulated (hostile bias, no bias, and friendly bias). We used signal detection measures to treat the identification results. RESULTS: ATD presence improved detection performance, especially under high task difficulty (night illumination). Identification sensitivity was greater for cued than uncued targets. The identification decision criterion for cued targets varied with the ATD identification bias but showed a "sluggish beta" effect. CONCLUSION: ATD helps soldiers detect and identify targets. The effects of biased ATD on identification should be considered with respect to the operational context. APPLICATION: Less-than-perfectly-reliable ATD is a useful detection aid for dismounted soldiers. Disclosure of known ATD identification bias to the operator may aid the identification process.

Beyond identity: incorporating system reliability information into an automated combat identification system.

OBJECTIVE: The aim of this study was to evaluate display formats for an automated combat identification (CID) aid. BACKGROUND: Verbally informing users of automation reliability improves reliance on automated CID systems. A display can provide reliability information in real time. METHOD: We developed and tested four visual displays that showed both target identity and system reliability information. Display type (pie, random mesh) and display proximity (integrated, separated) of identity and reliability information were manipulated. In Experiment 1, participants used the displays while engaging targets in a simulated combat environment. In Experiment 2, participants briefly viewed still scenes from the simulation. RESULTS: Participants relied on the automation more appropriately with the integrated display than with the separated display. Participants using the random mesh display showed greater sensitivity than those using a pie chart. However, in Experiment 2, the sensitivity effects were limited to lower reliability levels. CONCLUSION: The integrated display format and the random mesh display were the most effective displays tested. APPLICATION: We recommend the use of the integrated format and a random mesh display to indicate identity and reliability information with an automated CID system.

Viewpoint tethering for remotely operated vehicles: effects on complex terrain navigation and spatial awareness.

OBJECTIVE: The effect of viewpoint on the navigation of complex terrain and on spatial awareness was examined with the use of a simulated remotely operated vehicle. BACKGROUND: The ability to build terrain models in real time may soon allow remote vehicular control from any viewpoint. A virtual tether couples the viewpoint to the vehicle's position and orientation, but shows more of the terrain than a fully immersive egocentric display. In this sense, it provides visual momentum by providing a view that incorporates egocentric and exocentric qualities. METHOD: For this study, 12 participants navigated a simulated vehicle across complex virtual terrain using five different display viewpoints: egocentric, dynamic tether, rigid tether, 3-D exocentric, and 2-D exocentric. While navigating, participants had to avoid being seen by simulated enemy units. After the navigation task, participants' spatial awareness was assessed using a recognition task. RESULTS: The tethered displays minimized the time during which the participant's vehicle was visible to enemy positions. The egocentric display was more effective than exocentric displays (2-D or 3-D) for navigation, and the exocentric displays were more effective than egocentric for time seen during navigation and the recognition task. The tethered displays produced intermediate results for navigation and recognition. CONCLUSION: Viewpoint tethering produced the most effective displays for minimizing time seen, but tethered displays were less effective than egocentric and exocentric displays for navigation and recognition, respectively. APPLICATION: A tethered display is recommended for applications in which it is necessary to understand the relation of nearby locations to one's own location.

Revisiting confidence intervals for repeated measures designs.

Loftus and Masson (1994) proposed a method for computing confidence intervals (CIs) in repeated measures (RM) designs and later proposed that RM CIs for factorial designs should be based on number of observations rather than number of participants (Masson & Loftus, 2003). However, determining the correct number of observations for a particular effect can be complicated, given that its value depends on the relation between the effect and the overall design. To address this, we recently defined a general number-of-observations principle, explained why it obtains, and provided step-by-step instructions for constructing CIs for various effect types (Jarmasz & Hollands, 2009). In this note, we provide a brief summary of our approach.

Trust and reliance on an automated combat identification system.

OBJECTIVE: We examined the effects of aid reliability and reliability disclosure on human trust in and reliance on a combat identification (CID) aid. We tested whether trust acts as a mediating factor between belief in and reliance on a CID aid. BACKGROUND: Individual CID systems have been developed to reduce friendly fire incidents. However, these systems cannot positively identify a target that does not have a working transponder. Therefore, when the feedback is "unknown", the target could be hostile, neutral, or friendly. Soldiers have difficulty relying on this type of imperfect automation appropriately. METHOD: In manual and aided conditions, 24 participants completed a simulated CID task. The reliability of the aid varied within participants, half of whom were told the aid reliability level. We used the difference in response bias values across conditions to measure automation reliance. RESULTS: Response bias varied more appropriately with the aid reliability level when it was disclosed than when not. Trust in aid feedback correlated with belief in aid reliability and reliance on aid feedback; however, belief was not correlated with reliance. CONCLUSION: To engender appropriate reliance on CID systems, users should be made aware of system reliability. APPLICATION: The findings can be applied to the design of information displays for individual CID systems and soldier training.

Reference frame congruency in search-and-rescue tasks.

OBJECTIVE: Our aim was to investigate how the congruency between visual displays and auditory cues affects performance on various spatial tasks. BACKGROUND: Previous studies have demonstrated that spatial auditory cues, when combined with visual displays, can enhance performance and decrease workload. However, this facilitation was achieved only when auditory cues shared a common reference frame (RF) with the visual display. In complex and dynamic environments, such as airborne search and rescue (SAR), it is often difficult to ensure such congruency. METHOD: In a simulated SAR operation, participants performed three spatial tasks: target search, target localization, and target recall. The interface consisted of the camera view of the terrain from the aircraft-mounted sensor, a map of the area flown over, a joystick that controlled the sensor, and a mouse. Auditory cues were used to indicate target location. While flying in the scenario, participants searched for targets, identified their locations in one of two coordinate systems, and memorized their location relative to the terrain layout. RESULTS: Congruent cues produced the fastest and most accurate performance. Performance advantages were observed even with incongruent cues relative to neutral cues, and egocentric cues were more effective than exocentric cues. CONCLUSION: Although the congruent cues are most effective, in cases in which the same cue is used across spatial tasks, egocentric cues are a better choice than exocentric cues. APPLICATION: Egocentric auditory cues should be used in display design for tasks that involve RF transformations, such as SAR, air traffic control, and unmanned aerial vehicle operations.

Confidence intervals in repeated-measures designs: The number of observations principle.

Since the publication of Loftus and Masson's (1994) method for computing confidence intervals (CIs) in repeated-measures (RM) designs, there has been uncertainty about how to apply it to particular effects in complex factorial designs. Masson and Loftus (2003) proposed that RM CIs for factorial designs be based on number of observations rather than number of participants. However, determining the correct number of observations for a particular effect can be complicated, given the variety of effects occurring in factorial designs. In this paper the authors define a general "number of observations" principle, explain why it obtains, and provide step-by-step instructions for constructing CIs for various effect types. The authors illustrate these procedures with numerical examples.

Smooth rotation of 2-D and 3-D representations of terrain: an investigation into the utility of visual momentum.

OBJECTIVE: The potential advantage of visual momentum in the form of smooth rotation between two-dimensional (2-D) and three-dimensional (3-D) displays of geographic terrain was examined. BACKGROUND: The relative effectiveness of 2-D and 3-D displays is task dependent, leading to the need for multiple frames of reference as users switch tasks. The use of smooth rotation to provide visual momentum has received little scrutiny in the task-switching context. A cognitive model of the processes involved in switching viewpoints on a set of spatial elements is proposed. METHODS: In three experiments, participants judged the properties of two points placed on terrain depicted as 2-D or 3-D displays. Participants indicated whether Point A was higher than Point B, or whether Point B could be seen from Point A. Participants performed the two tasks in pairs of trials, switching tasks and displays within the pair. In the continuous transition condition the display dynamically rotated in depth from one display format to the other. In the discrete condition there was an instantaneous viewpoint shift that varied across experiments (Experiment 1: immediate; Experiment 2: delay; Experiment 3: preview). RESULTS: Performance after continuous transition was superior to that after discrete transition. CONCLUSION: The visual momentum provided by smooth rotation helped users switch tasks. APPLICATION: The use of dynamic transition is recommended when observers examine multiple views of terrain over time. The model may serve as a useful heuristic for designers. The results are pertinent to command and control, geological engineering, urban planning, and imagery analysis domains.