Virtual/augmented reality-based human-machine interface and interaction modes in airport control towers.

The concept of an innovative human-machine interface and interaction modes based on virtual and augmented reality technologies for airport control towers has been developed with the aim of increasing the human performances and situational awareness of air traffic control operators. By presenting digital information through see-through head-mounted displays superimposed over the out-of-the-tower view, the proposed interface should stimulate controllers to operate in a head-up position and, therefore, reduce the number of switches between a head-up and a head-down position even in low visibility conditions. This paper introduces the developed interface and describes the exercises conducted to validate the technical solutions developed, focusing on the simulation platform and exploited technologies, to demonstrate how virtual and augmented reality, along with additional features such as adaptive human-machine interface, multimodal interaction and attention guidance, enable a more natural and effective interaction in the control tower. The results of the human-in-the-loop real-time validation exercises show that the prototype concept is feasible from both an operational and technical perspective, the solution proves to support the air traffic controllers in working in a head-up position more than head-down even with low-visibility operational scenarios, and to lower the time to react in critical or alerting situations with a positive impact on the human performances of the user. While showcasing promising results, this study also identifies certain limitations and opportunities for refinement, aimed at further optimising the efficacy and usability of the proposed interface.

Rostering in Air Traffic Control: A Narrative Review.

Most Air Traffic Controllers (ATCOs) must cover uninterrupted work shifts for 24 h a day, seven days a week. The proper planning of a shift schedule requires consideration of at least three elements: the specific characteristics of the controller task, the physiological needs of the operator, and the definition of rest periods within rostering. We reviewed the literature for providing comprehensive guidance on the main requirements for the construction of a shift schedule for ATCOs. Our considerations are organized according to a rationale reflecting the most important criteria for the construction of the schedule: namely, the organization of rest periods conceptualized as intervals between cycles of shifts, intervals between individual shifts, and breaks within the shift. The suggested parameters could be used to construct shift schedules within a variation margin that depends on individual contexts of application.

How Neurophysiological Measures Can be Used to Enhance the Evaluation of Remote Tower Solutions.

New solutions in operational environments are often, among objective measurements, evaluated by using subjective assessment and judgment from experts. Anyhow, it has been demonstrated that subjective measures suffer from poor resolution due to a high intra and inter-operator variability. Also, performance measures, if available, could provide just partial information, since an operator could achieve the same performance but experiencing a different workload. In this study, we aimed to demonstrate: (i) the higher resolution of neurophysiological measures in comparison to subjective ones; and (ii) how the simultaneous employment of neurophysiological measures and behavioral ones could allow a holistic assessment of operational tools. In this regard, we tested the effectiveness of an electroencephalography (EEG)-based neurophysiological index (WEEG index) in comparing two different solutions (i.e., Normal and Augmented) in terms of experienced workload. In this regard, 16 professional air traffic controllers (ATCOs) have been asked to perform two operational scenarios. Galvanic Skin Response (GSR) has also been recorded to evaluate the level of arousal (i.e., operator involvement) during the two scenarios execution. NASA-TLX questionnaire has been used to evaluate the perceived workload, and an expert was asked to assess performance achieved by the ATCOs. Finally, reaction times on specific operational events relevant for the assessment of the two solutions, have also been collected. Results highlighted that the Augmented solution induced a local increase in subjects performance (Reaction times). At the same time, this solution induced an increase in the workload experienced by the participants (WEEG). Anyhow, this increase is still acceptable, since it did not negatively impact the performance and has to be intended only as a consequence of the higher engagement of the ATCOs. This behavioral effect is totally in line with physiological results obtained in terms of arousal (GSR), that increased during the scenario with augmentation. Subjective measures (NASA-TLX) did not highlight any significant variation in perceived workload. These results suggest that neurophysiological measure provide additional information than behavioral and subjective ones, even at a level of few seconds, and its employment during the pre-operational activities (e.g., design process) could allow a more holistic and accurate evaluation of new solutions.

Human-Machine Interaction Assessment by Neurophysiological Measures: A Study on Professional Air Traffic Controllers.

This study aims at investigating the possibility to employ neurophysiological measures to assess the humanmachine interaction effectiveness. Such a measure can be used to compare new technologies or solutions, with the final purpose to enhance operator's experience and increase safety. In the present work, two different interaction modalities (Normal and Augmented) related to Air Traffic Management field have been compared, by involving 10 professional air traffic controllers in a control tower simulated environment. Experimental task consisted in locating aircrafts in different airspace positions by using the sense of hearing. In one modality (i.e. "Normal"), all the sound sources (aircrafts) had the same amplification factor. In the "Augmented" modality, the amplification factor of the sound sources located along the participant head sagittal axis was increased, while the intensity of sound sources located outside this axis decreased. In other words, when the user oriented his head toward the aircraft position, the related sound was amplified. Performance data, subjective questionnaires (i.e. NASA-TLX) and neurophysiological measures (i.e. EEG-based) related to the experienced workload have been collected. Results showed higher significant performance achieved by the users during the "Augmented" modality with respect to the "Normal" one, supported by a significant decreasing in experienced workload, evaluated by using EEG-based index. In addition, Performance and EEG-based workload index showed a significant negative correlation. On the contrary, subjective workload analysis did not show any significant trend. This result is a demonstration of the higher effectiveness of neurophysiological measures with respect to subjective ones for Human-Computer Interaction assessment.

Reliability over time of EEG-based mental workload evaluation during Air Traffic Management (ATM) tasks.

Machine-learning approaches for mental workload (MW) estimation by using the user brain activity went through a rapid expansion in the last decades. In fact, these techniques allow now to measure the MW with a high time resolution (e.g. few seconds). Despite such advancements, one of the outstanding problems of these techniques regards their ability to maintain a high reliability over time (e.g. high accuracy of classification even across consecutive days) without performing any recalibration procedure. Such characteristic will be highly desirable in real world applications, in which human operators could use such approach without undergo a daily training of the device. In this work, we reported that if a simple classifier is calibrated by using a low number of brain spectral features, between those ones strictly related to the MW (i.e. Frontal and Occipital Theta and Parietal Alpha rhythms), those features will make the classifier performance stable over time. In other words, the discrimination accuracy achieved by the classifier will not degrade significantly across different days (i.e. until one week). The methodology has been tested on twelve Air Traffic Controls (ATCOs) trainees while performing different Air Traffic Management (ATM) scenarios under three different difficulty levels.

ASTEF: a simple tool for examining fixations.

In human factors and ergonomics research, the analysis of eye movements has gained popularity as a method for obtaining information concerning the operator's cognitive strategies and for drawing inferences about the cognitive state of an individual. For example, recent studies have shown that the distribution of eye fixations is sensitive to variations in mental workload---dispersed when workload is high, and clustered when workload is low. Spatial statistics algorithms can be used to obtain information about the type of distribution and can be applied over fixations recorded during small epochs of time to assess online changes in the level of mental load experienced by the individuals. In order to ease the computation of the statistical index and to encourage research on the spatial properties of visual scanning, A Simple Tool for Examining Fixations has been developed. The software application implements functions for fixation visualization, management, and analysis, and includes a tool for fixation identification from raw gaze point data. Updated information can be obtained online at www.astef.info, where the installation package is freely downloadable.

Procedural errors in air traffic control: effects of traffic density, expertise, and automation.

INTRODUCTION: Air traffic management requires operators to frequently shift between multiple tasks and/or goals with different levels of accomplishment. Procedural errors can occur when a controller accomplishes one of the tasks before the entire operation has been completed. The present study had two goals: first, to verify the occurrence of post-completion errors in air traffic control (ATC) tasks; and second, to assess effects on performance of medium term conflict detection (MTCD) tools. METHODS: There were 18 military controllers who performed a simulated ATC task with and without automation support (MTCD vs. manual) in high and low air traffic density conditions. During the task, which consisted of managing several simulated flights in an enroute ATC scenario, a trace suddenly disappeared "after" the operator took the aircraft in charge, "during" the management of the trace, or "before" the pilot's first contact. RESULTS: In the manual condition, only the fault type "during" was found to be significantly different from the other two. On the contrary, when in the MTCD condition, the fault type "after" generated significantly less errors than the fault type "before." Additionally, automation was found to affect performance of junior controllers, whereas seniors' performance was not affected. DISCUSSION: Procedural errors can happen in ATC, but automation can mitigate this effect. Lack of benefits for the "before" fault type may be due to the fact that operators extend their reliance to a part of the task that is unsupported by the automated system.