Facilitating the Work of Unmanned Aerial Vehicle Operators Using Artificial Intelligence: An Intelligent Filter for Command-and-Control Maps to Reduce Cognitive Workload.

OBJECTIVE: Evaluating the ability of a Gibsonian-inspired artificial intelligence (AI) algorithm to reduce the cognitive workloads of military Unmanned Aerial Vehicle (UAV) operators. BACKGROUND: Military UAV operators use the command-and-control (C2) map for developing mission-relevant situation awareness (SA). Yet C2 maps are overloaded with information, mostly irrelevant to the mission, causing operators to neglect the map altogether. To reduce irrelevant information, an intelligent filtering algorithm was developed. Here we evaluate its effectiveness in reducing operators' cognitive workloads. METHOD: Two-stage operational scenarios were conducted with professional ex-military UAV operators, using two filter protocols and a no-filter control. High-end real-time techniques were used to continuously assess workload from muscle behavior and machine learning models. RESULTS: Lower cognitive workload was found when applying the algorithm's protocols, especially when fatigue started to accumulate (Stage II). However, concerns about the quality of SA arose. CONCLUSION: The algorithm was positively evaluated for its ability to reduce operators' cognitive workloads. More evaluations of operators' SA are required. APPLICATION: The algorithm demonstrates the possibility of integrating AI to improve human performance in complex systems, and can be applied to other domains where spatial-temporal information needs to be contextually filtered in real time.

Evaluating levels of automation with different feedback modes in an assistive robotic table clearing task for eldercare.

This paper focuses on how the autonomy level of an assistive robot that offers support for older adults in a daily task and its feedback affect the interaction. Identifying the level of automation (LOA) that prioritizes older adults' preferences while avoiding passiveness and sedentariness is challenging. The feedback mode should match the cognitive and perceptual capabilities of older adults and the LOA. We characterized three LOAs and paired them with two modes of feedback in a human-robot collaborative task. Twenty-seven older adults participated in evaluating the LOA-feedback variations in a mixed experimental design, utilizing an experimental setup of an assistive robot in a table clearing task. The quality of the interaction was evaluated with objective and subjective measures. The combination of high LOA with voice feedback improved the overall interaction when compared to other LOA and feedback combinations. This study emphasizes the importance of appropriate coupling of LOA and feedback for successful interaction of the older adults with an assistive robot.

Is It Personal? The Impact of Personally Relevant Robotic Failures (PeRFs) on Humans' Trust, Likeability, and Willingness to Use the Robot.

In three laboratory experiments, we examine the impact of personally relevant failures (PeRFs) on users' perceptions of a collaborative robot. PeR is determined by how much a specific issue applies to a particular person, i.e., it affects one's own goals and values. We hypothesized that PeRFs would reduce trust in the robot and the robot's Likeability and Willingness to Use (LWtU) more than failures that are not personal to participants. To achieve PeR in human-robot interaction, we utilized three different manipulation mechanisms: (A) damage to property, (B) financial loss, and (C) first-person versus third-person failure scenarios. In total, 132 participants engaged with a robot in person during a collaborative task of laundry sorting. All three experiments took place in the same experimental environment, carefully designed to simulate a realistic laundry sorting scenario. Results indicate that the impact of PeRFs on perceptions of the robot varied across the studies. In experiments A and B, the encounters with PeRFs reduced trust significantly relative to a no failure session. But not entirely for LWtU. In experiment C, the PeR manipulation had no impact. The work highlights challenges and adjustments needed for studying robotic failures in laboratory settings. We show that PeR manipulations affect how users perceive a failing robot. The results bring about new questions regarding failure types and their perceived severity on users' perception of the robot. Putting PeR aside, we observed differences in the way users perceive interaction failures compared (experiment C) to how they perceive technical ones (A and B).

Expect the Unexpected: Leveraging the Human-Robot Ecosystem to Handle Unexpected Robot Failures.

Unexpected robot failures are inevitable. We propose to leverage socio-technical relations within the human-robot ecosystem to support adaptable strategies for handling unexpected failures. The Theory of Graceful Extensibility is used to understand how characteristics of the ecosystem can influence its ability to respond to unexpected events. By expanding our perspective from Human-Robot Interaction to the Human-Robot Ecosystem, adaptable failure-handling strategies are identified, alongside technical, social and organizational arrangements that are needed to support them. We argue that robotics and HRI communities should pursue more holistic approaches to failure-handling, recognizing the need to embrace the unexpected and consider socio-technical relations within the human robot ecosystem when designing failure-handling strategies.

Combining cognitive work analysis and empirical evaluations to understand map use by operators of small carry-on unmanned aerial systems.

Operating a small carry-on unmanned aerial system (UAS) alone is challenging. Research on facilitating single-operator work has focused mainly on payload operation and health monitoring. Little focus has been given to mission-related aspects and how the command and control (C2) map display contributes to mission accomplishment. This study uses cognitive work analysis (CWA) to describe the operational work of the mission operator of a Skylark miniature UAS system. Three CWA phases were conducted - work domain analysis, control task analysis and strategy analysis - providing a rich framework of operational mission phases, task components, processes and the physical interface-objects in use. These representations highlight the operators' extensive use of the C2 map during all mission phases, for all object-related processes. To further enhance the outcomes of the CWA, and prior to outlining specific design requirements, an empirical investigation was conducted in which the eye movements of five experienced operators were obtained during a simulated mission. The empirical results confirm and further specify the work patterns that operators adopt. Quantitative analysis shows operators' extensive focus on the map, especially during mission-critical phases. These analyses led to the conclusion that a significant change in the way operators interact with the C2 map, or alternative designs to enhance map-based information utilization, should be applied. Insights drawn from this analysis can be applied to other aerial surveillance work domains, and adding empirical evaluations is helpful to further refine and reinforce the CWA outcomes.

Pedestrians' Understanding of a Fully Autonomous Vehicle's Intent to Stop: A Learning Effect Over Time.

This study explored pedestrians' understanding of Fully Autonomous Vehicles (FAVs) intention to stop and what influences pedestrians' decision to cross the road over time, i.e., learnability. Twenty participants saw fixed simulated urban road crossing scenes with a single FAV on the road as if they were pedestrians intending to cross. Scenes differed from one another in the FAV's, distance from the crossing place, its physical size, and external Human-Machine Interfaces (e-HMI) message by background color (red/green), message type (status/advice), and presentation modality (text/symbol). Eye-tracking data and decision measurements were collected. Results revealed that pedestrians tend to look at the e-HMI before making their decision. However, they did not necessarily decide according to the e-HMIs' color or message type. Moreover, when they complied with the e-HMI proposition, they tended to hesitate before making the decision. Overall, a learning effect over time was observed in all conditions regardless of e- HMI features and crossing context. Findings suggest that pedestrians' decision making depends on a combination of the e-HMI implementation and the car distance. Moreover, since the learning curve exists in all conditions and has the same proportion, it is critical to design an interaction that would encourage higher probability of compatible decisions from the first phase. However, to extend all these findings, it is necessary to further examine dynamic situations.

Pedestrian distraction: The effects of road environment complexity and age on pedestrian's visual attention and crossing behavior.

INTRODUCTION: Little is known about how characteristics of the environment affect pedestrians' road crossing behavior. METHOD: In this work, the effect of typical urban visual clutter created by objects and elements in the road proximity (e.g., billboards) on adults and children (aged 9-13) road crossing behavior was examined in a controlled laboratory environment, utilizing virtual reality scenarios projected on a large dome screen. RESULTS: Divided into three levels of visual load, results showed that high visual load affected children's and adults' road crossing behavior and visual attention. The main effect on participants' crossing decisions was seen in missed crossing opportunities. Children and adults missed more opportunities to cross the road when exposed to more cluttered road environments. An interaction with age was found in the dispersion of the visual attention measure. Children, 9-10 and 11-13 years old, had a wider spread of gazes across the scene when the environment was highly loaded-an effect not seen with adults. However, unexpectedly, no other indication of the deterring effect was found in the current study. Still, according to the results, it is reasonable to assume that busier road environments can be more hazardous to adult and child pedestrians. Practical Applications: In that context, it is important to further investigate the possible distracting effect of causal objects in the road environment on pedestrians, and especially children. This knowledge can help to create better safety guideline for children and assist urban planners in creating safer urban environments.

Understanding complex traffic road scenes: The case of child-pedestrians' hazard perception.

INTRODUCTION: Understanding the shortcomings of child-pedestrians in evaluating traffic situations may contribute to producing intervention techniques that may increase their awareness to potential hazards as well as inform and inspire designers of autonomous vehicle and infrastructure systems to deal with the complications of crossing pedestrians. METHOD: The present work examined pedestrians' hazard-perception (HP) skills in complex traffic scenes. Two experiments explored how pedestrians' HP abilities vary with age and experience. In the first, adults and youngsters (7-13-year-olds) were presented with pairs of photographs displaying traffic situations and instructed to compare between the hazard levels of the two. Findings revealed a marked trend where experienced-adults tended to rate photographs depicting field of view partially obscured by parked vehicles as more hazardous. Moreover, adults tended to rate photographs depicting vehicles closer to the crossing site as more hazardous. Lastly, adults tended to rate photographs depicting complex configurations like traffic circles, as more hazardous than T-junctions. RESULTS: Findings suggested that youngsters may be highly influenced by cueing. Next, pedestrians' HP was tested using a crossing decision task. Participants observed traffic scenes presented in a dynamic simulated environment of an urban road from a pedestrian's perspective and pressed a response button whenever they assumed it was safe to cross. Compared to experienced-adults and 7-8-year-olds, 9-13-year-olds presented a less decisive performance. Compared to previous findings regarding simpler road crossing configurations, most participants, regardless of age, related more to the approaching vehicles and presence of a pedestrian crossing while refraining from addressing the road configuration. Implications for road-safety are discussed.

Understanding and Resolving Failures in Human-Robot Interaction: Literature Review and Model Development.

While substantial effort has been invested in making robots more reliable, experience demonstrates that robots operating in unstructured environments are often challenged by frequent failures. Despite this, robots have not yet reached a level of design that allows effective management of faulty or unexpected behavior by untrained users. To understand why this may be the case, an in-depth literature review was done to explore when people perceive and resolve robot failures, how robots communicate failure, how failures influence people's perceptions and feelings toward robots, and how these effects can be mitigated. Fifty-two studies were identified relating to communicating failures and their causes, the influence of failures on human-robot interaction (HRI), and mitigating failures. Since little research has been done on these topics within the HRI community, insights from the fields of human computer interaction (HCI), human factors engineering, cognitive engineering and experimental psychology are presented and discussed. Based on the literature, we developed a model of information processing for robotic failures (Robot Failure Human Information Processing, RF-HIP), that guides the discussion of our findings. The model describes the way people perceive, process, and act on failures in human robot interaction. The model includes three main parts: (1) communicating failures, (2) perception and comprehension of failures, and (3) solving failures. Each part contains several stages, all influenced by contextual considerations and mitigation strategies. Several gaps in the literature have become evident as a result of this evaluation. More focus has been given to technical failures than interaction failures. Few studies focused on human errors, on communicating failures, or the cognitive, psychological, and social determinants that impact the design of mitigation strategies. By providing the stages of human information processing, RF-HIP can be used as a tool to promote the development of user-centered failure-handling strategies for HRIs.

Supervising and Controlling Unmanned Systems: A Multi-Phase Study with Subject Matter Experts.

Proliferation in the use of Unmanned Aerial Systems (UASs) in civil and military operations has presented a multitude of human factors challenges; from how to bridge the gap between demand and availability of trained operators, to how to organize and present data in meaningful ways. Utilizing the Design Research Methodology (DRM), a series of closely related studies with subject matter experts (SMEs) demonstrate how the focus of research gradually shifted from "how many systems can a single operator control" to "how to distribute missions among operators and systems in an efficient way". The first set of studies aimed to explore the modal number, i.e., how many systems can a single operator supervise and control. It was found that an experienced operator can supervise up to 15 UASs efficiently using moderate levels of automation, and control (mission and payload management) up to three systems. Once this limit was reached, a single operator's performance was compared to a team controlling the same number of systems. In general, teams led to better performances. Hence, shifting design efforts toward developing tools that support teamwork environments of multiple operators with multiple UASs (MOMU). In MOMU settings, when the tasks are similar or when areas of interest overlap, one operator seems to have an advantage over a team who needs to collaborate and coordinate. However, in all other cases, a team was advantageous over a single operator. Other findings and implications, as well as future directions for research are discussed.

Can child-pedestrians' hazard perception skills be enhanced?

OBJECTIVE: Traffic collisions yield a substantial rate of morbidity and injury among child-pedestrians. We explored the formation of an innovative hazard perception training intervention - Child-pedestrians Anticipate and Act Hazard Perception Training (CA(2)HPT). Training was based upon enhancing participants' ability to anticipate potential hazards by exposing them to an array of traffic scenes viewed from different angles. METHOD: Twenty-four 7-9-year-olds have participated. Trainees underwent a 40-min intervention of observing typical residential traffic scenarios in a simulated dome projection environment while engaging in a hazard detection task. Trainees were encouraged to note differences between the scenarios presented to them from separate angles (a pedestrian's point-of-view and a higher perspective angle). Next, trainees and control group members were required to perform crossing decision tasks. RESULTS: Trainees were found to be more aware of potential hazards related to restricted field of view relative to control. CONCLUSIONS: Child pedestrians are responsive to training and actively detecting materialized hazards may enrich child-pedestrians' ability to cross roads.

Driving speed of young novice and experienced drivers in simulated hazard anticipation scenes.

OBJECTIVE: In this study, we aimed to demonstrate analysis methods that are sensitive to speed-related differences between experienced and young novice drivers. These differences may be linked to determining which group is better at anticipating hazards. BACKGROUND: Awareness of hazardous situations, especially potential ones, is a major discriminator between experienced and young novice drivers who tend to misidentify potential hazards in the traffic environment. METHOD: Experienced and young novice drivers were asked to drive a sequence of 14 scenarios in a driving simulator. Scenarios were created in two city areas, residential and business district, and included various types of hazards. Group homogeneity of speed for each group of drivers was computed for each scenario, and two business district scenarios were subjected to piecewise linear regression analysis. RESULTS: Group homogeneity analysis showed consistent and significant experience-based differences across all scenarios, revealing that the experienced drivers as a group were more homogenous in choosing their driving speed. Differences between groups were larger in the business district where speed was less restricted. Piecewise linear regression analysis revealed that experienced drivers approached uncontrolled intersections by slowing down and responded earlier to materialized events. CONCLUSION: Young novice drivers were more likely than experienced drivers to choose diverse values of speed at any given road section, presumably due to their poor awareness of potential and hidden hazards. Unlike other analysis methods, it is argued that group homogeneity of speed is a more sensitive measurement to reveal these gaps. APPLICATION: Speed management could be the basis of future hazard anticipation simulator assessments.

Evaluation of the attention network test using vibrotactile stimulations.

We report a vibrotactile version of the attention network test (ANT)-the tactile ANT (T-ANT). It has been questioned whether attentional components are modality specific or not. The T-ANT explores alertness, orienting, cognitive control, and their relationships, similar to its visual counterpart, in the tactile modality. The unique features of the T-ANT are in utilizing stimuli on a single plane-the torso-and replacing the original imperative flanker task with a tactile Simon task. Subjects wore a waist belt mounted with two vibrotactile stimulators situated on the back and positioned to the right and left of the spinal column. They responded by pressing keys with their right or left hand in reaction to the type of vibrotactile stimulation (pulsed/continuous signal). On a single trial, an alerting tone was followed by a short tactile (informative/noninformative) peripheral cue and an imperative tactile Simon task target. The T-ANT was compared with a variant of the ANT in which the flanker task was replaced with a visual Simon task. Experimental data showed effects of orienting over control only when the peripheral cues were informative. In contrast to the visual task, interactions between alertness and control or alertness and orienting were not found in the tactile task. A possible rationale for these results is discussed. The T-ANT allows examination of attentional processes among patients with tactile attentional deficits and patients with eyesight deficits who cannot take part in visual tasks. Technological advancement would enable implementation of the T-ANT in brain-imaging studies.

The combination of short rest and energy drink consumption as fatigue countermeasures during a prolonged drive of professional truck drivers.

One of the major concerns for professional drivers is fatigue. Many studies evaluated specific fatigue countermeasures, in many cases comparing the efficiency of each method separately. The present study evaluated the effectiveness of rest areas combined with consumption of energy drinks on professional truck drivers during a prolonged simulated drive. Fifteen professional truck drivers participated in three experimental sessions: control-drivers were asked to drink 500 ml of a placebo drink prior to the beginning of the drive. Energy drink-drivers were asked to drink 500 ml of an energy drink containing 160 mg of caffeine prior to the beginning of the drive, and an Energy drink+Rest session--where the drivers were asked to drink 500 ml of an energy drink prior to driving, and rest for 10 min at a designated rest area zone 100 min into the drive. For all sessions, driving duration was approximately 150 min and consisted of driving on a monotonous, two-way rural road. In addition to driving performance measures, subjective measures, and heart rate variability were obtained. Results indicated that consumption of an energy drink (in both sessions) facilitated lower lane position deviations and reduced steering wheel deviations during the first 80-100 min of the drive relative to the control sessions. Resting after 100 min of driving, in addition to the energy drink that was consumed before the drive, enabled the drivers to maintain these abilities throughout the remainder of the driving session. Practical applications: Practical applications arising from the results of this research may give indication on the possible added value of combining fatigue counter measures methods during a prolonged drive and the importance of the timing of the use for each method.

Coding warnings without interfering with dismounted soldiers' missions.

OBJECTIVES: Warnings are an effective way to communicate hazard, yet they can also increase task demand when presented to operators involved in real-world tasks. Furthermore, in military-related tasks warnings are often given in codes to avoid counter-intelligence, which may foster additional working memory load. BACKGROUND: Adherence to warnings in the military domain is crucial to promote safety and reduce accidents and injuries. The empirical question arises as to how aspects of coding the warning may interfere with the primary task the individual is currently performing and vice versa. METHOD: Six experimental conditions were designed to assess how warning-code storage format, response format, and increasing working memory demand (retention) affected both performance on the primary task and the rate of compliance to warnings, considered here as the secondary task. RESULTS: Results revealed that the combination of warning-code storage and response format affected compliance rate and the highest compliance occurred when warnings were presented as pictorials and responses were coded verbally. Contrary to the proposed hypotheses, warning storage format did not affect performance on the primary task, which was only affected by the level of working memory demand. Thus, the intra-modal warning storages did not interfere with the visual/spatial nature of the primary operational task. However, increase in working memory demand, by increasing the number of memorized warning codes, had an effect on both compliance rate and primary task performance. CONCLUSIONS: Rather than warning code storage alone, it is the coupling of warning storage and response format that has the most significant effect on compliance.

In touch with the Simon effect.

Cognitive control has been extensively studied using the auditory and visual modalities. In the current study, a tactile version of the Simon task was created in order to test control mechanisms in a modality that was less studied, to provide comparative and new information. A significant Simon effect--reaction time gap between congruent (i.e., stimulus and response in the same relative location) and incongruent (i.e., stimulus and response in opposite locations) stimuli--provided grounds to further examine both general and tactile-specific aspects of cognitive control in three experiments. By implementing a neutral condition and conducting sequential and distributional analysis, the present study: (a) supports two different independent mechanisms of cognitive control--reactive control and proactive control; (b) reveals facilitation and interference within the tactile Simon effect; and (c) proposes modality differences in activation and processing of the spatially driven stimulus-response association.

Formation and Evaluation of Act and Anticipate Hazard Perception Training (AAHPT) intervention for young novice drivers.

OBJECTIVE: Young novice drivers' poor hazard perception (HP) skills are a prominent cause for their overinvolvement in traffic crashes. HP, the ability to read the road and anticipate forthcoming events, is receptive to training. This study explored the formation and evaluation of a new HP training intervention-the Act and Anticipate Hazard Perception Training (AAHPT), which is based upon exposing young novice drivers to a vast array of actual traffic hazards, aiming to enhance their ability to anticipate potential hazards during testing. METHOD: Forty young novices underwent one of 3 AAHPT intervention modes (active, instructional, or hybrid) or a control group. Active members observed video-based traffic scenes and were asked to press a response button each time they detected a hazard. Instructional members underwent a tutorial that included both written material and video-based examples regarding HP. Hybrid members received a condensed theoretical component followed by a succinct active component. Control was presented with a road safety tutorial. Approximately one week later, participants performed a hazard perception test (HPT), during which they observed other movies and pressed a response button each time they detected a hazard. Twenty-one experienced drivers also performed the HPT and served as a gold standard for comparison. RESULTS: Overall, the active and hybrid modes were more aware of potential hazards relative to the control. CONCLUSIONS: Inclusion of an active-practical component generates an effective intervention. Using several evaluation measurements aids performance assessment process. Advantages of each of the training methodologies are discussed. Supplemental materials are available for this article. Go to the publisher's online edition of Traffic Injury Prevention to view the supplemental file.

Exploring the effects of driving experience on hazard awareness and risk perception via real-time hazard identification, hazard classification, and rating tasks.

This study investigated the effects of driving experience on hazard awareness and risk perception skills. These topics have previously been investigated separately, yet a novel approach is suggested where hazard awareness and risk perception are examined concurrently. Young, newly qualified drivers, experienced drivers, and a group of commercial drivers, namely, taxi drivers performed three consecutive tasks: (1) observed 10 short movies of real-world driving situations and were asked to press a button each time they identified a hazardous situation; (2) observed one of three possible sub-sets of 8 movies (out of the 10 they have seen earlier) for the second time, and were asked to categorize them into an arbitrary number of clusters according to the similarity in their hazardous situation; and (3) observed the same sub-set for a third time and following each movie were asked to rate its level of hazardousness. The first task is considered a real-time identification task while the other two are performed using hindsight. During it participants' eye movements were recorded. Results showed that taxi drivers were more sensitive to hidden hazards than the other driver groups and that young-novices were the least sensitive. Young-novice drivers also relied heavily on materialized hazards in their categorization structure. In addition, it emerged that risk perception was derived from two major components: the likelihood of a crash and the severity of its outcome. Yet, the outcome was rarely considered under time pressure (i.e., in real-time hazard identification tasks). Using hindsight, when drivers were provided with the opportunity to rate the movies' hazardousness more freely (rating task) they considered both components. Otherwise, in the categorization task, they usually chose the severity of the crash outcome as their dominant criterion. Theoretical and practical implications are discussed.

The effect of system aesthetics on trust, cooperation, satisfaction and annoyance in an imperfect automated system.

Lack of system reliability has been repeatedly identified as a factor that decreases trust. However, aesthetics has an important role in the development of trust. Most of the research concerning the connection between aesthetics and trust focused on mobile commerce and websites while very little has been done in examining aesthetics in automated systems. This study integrated aesthetics manipulations into an imperfect in-vehicle automation system and focused on the power of aesthetics to decrease the negative effects of errors on trust, satisfaction, annoyance, and human-automation cooperation perceptions. Participants used the navigation system in either 100% or 85% accuracy levels with an aesthetic or non aesthetic system (4 conditions). In both aesthetic and non aesthetic systems, perceptions of trust, satisfaction and human automation cooperation were decreased in the imperfect system compared to the perfect one. However, in the annoyance rating, this trend was found only in the aesthetic system while in the non-aesthetic system no difference was found between the two levels of accuracy. This single effect may indicate upon the possibility that in automated systems aesthetics affects trust and satisfaction more moderately compared to mobile commerce applications and websites. However, more research is needed to assess this assumption.