Attentional warnings caused by driver monitoring systems: How often do they appear and how well are they understood?

The present study investigated the effects of a driver monitoring system that triggers attention warnings in case distraction is detected. Based on the EuroNCAP protocol, distraction could either be long glances away from the forward roadway (≥3s) or visual attention time sharing (>10 cumulative seconds within a 30 s time interval). In a series of manual driving simulator drives, 30 participants completed both driving related tasks (e.g., changing multiple lanes in dense traffic) and non-driving related tasks (e.g., infotainment operations). Results of warning frequencies revealed that visual attention time sharing warnings occurred more frequently than long distraction warnings. Moreover, there was a large number of attention warnings during driving related tasks. Results also revealed that participants' mental models tended to be less accurate when it came to understanding of the visual attention time sharing warnings as compared to the long distraction warnings, which were understood more accurately. Based on these observations, the work discusses the applicability and design of driver monitoring warnings.

Test procedure for the evaluation of partially automated driving HMI including driver monitoring systems in driving simulation.

The proposed test procedure presents an approach for the evaluation of the usability of partial automated driving HMI including driver monitoring systems in driving simulation. This procedure is based on a definition of requirements that a Level 2 HMI and its included driver monitoring system must fulfill in order to guarantee that the drivers understand their responsibilities of continuously monitoring the driving environment and the status of the partial automated driving system. These requirements are used to define the evaluation criteria that have to be validated in the test as well as the use cases in which these criteria can be assessed. The result is a detailed and comprehensive test guide including the specification of the test drives, the necessary instructions, the test environment and the recruiting criteria for the test sample.•Evaluation of usability aspects of level 2 automated driving HMI including driver monitoring systems•Based on the definition of requirements for L2 HMI•Test guide including the definition of use cases, evaluation criteria and testing conditions in driving simulation.

The impact of speech-based assistants on the driver's cognitive distraction.

Speech-based interfaces can be a promising alternative and/or addition to visual-manual interfaces since they reduce visual-manual distraction while driving. However, there are also findings indicating that speech-based assistants may be a source of cognitive distraction. The aim of this experiment was to quantify drivers' cognitive distraction while interacting with speech-based assistants. Therefore, 31 participants performed a simulated driving task and a detection response task (DRT). Concurrently they either sent text-messages via speech-based assistants (Siri, Google Assistant, or Alexa) or completed an arithmetic task (OSPAN). In a multifactorial approach, following Strayer et al. (2017), cognitive distraction was then assessed through performance in the DRT, the driving speed, the task completion time and self-report measures. The cognitive distraction associated with speech-based assistants was compared to the OSPAN task and a baseline condition without a secondary task. Participants reacted faster and more accurately to the DRT in the baseline condition compared to the speech conditions. The performance in the speech conditions was significantly better than in the OSPAN task. However, driving speed did not significantly differ between the experimental conditions. Results from the NASA-TLX indicate that speech-based tasks were more demanding than the baseline but less demanding than the OSPAN task. The task completion times revealed significant differences between speech-based assistants. Sending messages took longest with the Google Assistant. Referring to the findings by Strayer et al. (2017), we conclude that nowadays speech-based assistants are associated with a rather moderate than high level of cognitive distraction. Nonetheless, we point towards the need to assess the effects of human-machine interaction via speech-based interfaces due to their potential for cognitive distraction.

The box task - a method for assessing in-vehicle system demand.

The use of advanced in-vehicle information systems (IVIS) and other complex devices such as smartphones while driving can lead to driver distraction, which, in turn, increases safety-critical event risk. Therefore, using methods for measuring driver distraction caused by IVIS is crucial when developing new in-vehicle systems. In this paper, we present the setup and implementation of the Box Task combined with a Detection Response Task (BT+DRT) as a tool to assess visual-manual and cognitive distraction effects. The BT+DRT represents a low-cost and easy-to-use method which can be easily implemented by researchers in laboratory settings and which was validated in previous research. Moreover, at the end of this paper we describe the experimental procedure, the data analysis and discuss potential modifications of the method.•The setup and implementation of the Box Task combined with a Detection Response Task (BT+DRT) is described.•The method allows for measuring visual-manual and cognitive distraction of drivers.•The BT+DRT is a cost-effective and easy-to-use method that can be implemented in laboratory settings or driving simulators.

Measuring driver distraction - Evaluation of the box task method as a tool for assessing in-vehicle system demand.

Several tools have been developed over the past twenty years to assess the degree of driver distraction caused by secondary task engagement. A relatively new and promising method in this area is the box task combined with a detection response task (BT + DRT). However, no evaluation regarding the BT's sensitivity currently exists. Thus, the aim of the present study was to evaluate the BT + DRT by comparing its sensitivity to the sensitivity of already established methods. Twenty-nine participants engaged in several artificial and realistic secondary tasks while either performing the BT + DRT, the Lane Change Test (LCT), or driving through a simple course in a simulator. The results showed that the BT parameters (especially the standard deviation of box position and size) were sensitive to differences in demand across the visual-manual secondary tasks. This was comparable to what was found with the LCT. Surprisingly, the BT performance measures were more sensitive than those of the driving simulation task. The BT + DRT also captured cognitive distraction effects with the integration of the DRT. Hence, the BT + DRT could be a cost-effective method to assess in-vehicle system demand. However, further investigations are necessary to better understand the potential of the BT method.

Using European naturalistic driving data to assess secondary task engagement when stopped at a red light.

PROBLEM: Some evidence exists that drivers choose to engage in secondary tasks when the driving demand is low (e.g., when the car is stopped). While such a behavior might generally be considered as rather safe, it could be argued that the associated diversion of attention away from the road still leads to a reduction of situational awareness, which might increase collision risk once the car regains motion. This is especially relevant for texting, which is associated with considerable eyes-off-the-road-time. Nonetheless, it seems that previous research has barely addressed the actual engagement in secondary tasks while waiting at a red light (as compared to just addressing the tasks' mere prevalence). OBJECTIVE: The present study investigated secondary task engagement while stopped at a red light using European naturalistic driving data collected through the UDRIVE project. Attention was given to the whole engagement process, including simple prevalence and the tasks' relation (in terms of start/end) to the red light period. Moreover, given that texting is one of the most problematic forms of distraction, it was characterized in more detail regarding glance behavior. METHOD: Videos of 804 red light episodes from 159 drivers were annotated. Glance behavior was also coded for a sub-set of 75 texting events and their matched baselines. Results, conclusions and practical applications: Drivers engaged in at least one secondary task across almost half of the annotated red light episodes. Drivers who texted while stopped spent most of the time looking at their cell phone. Consequently, drivers might not have been prepared for potentially unexpected events once the light turned green. Further, drivers concluded texting a considerable number of times well after the red light period, which has potential implications for traffic safety.

Test procedure for evaluating the human-machine interface of vehicles with automated driving systems.

Objective: The human-machine interface (HMI) is a crucial part of every automated driving system (ADS). In the near future, it is likely that-depending on the operational design domain (ODD)-different levels of automation will be available within the same vehicle. The capabilities of a given automation level as well as the operator's responsibilities must be communicated in an appropriate way. To date, however, there are no agreed-upon evaluation methods that can be used by human factors practitioners as well as researchers to test this. Methods: We developed an iterative test procedure that can be applied during the product development cycle of ADS. The test procedure is specifically designed to evaluate whether minimum requirements as proposed in NHTSA's automated vehicle policy are met. Results: The proposed evaluation protocol includes (a) a method to identify relevant use cases for testing on the basis of all theoretically possible steady states and mode transitions of a given ADS; (b) an expert-based heuristic assessment to evaluate whether the HMI complies with applicable norms, standards, and best practices; and (c) an empirical evaluation of ADS HMIs using a standardized design for user studies and performance metrics. Conclusions: Each can be used as a stand-alone method or in combination to generate objective, reliable, and valid evaluations of HMIs, focusing on whether they meet minimum requirements. However, we also emphasize that other evaluation aspects such as controllability, misuse, and acceptance are not within the scope of the evaluation protocol.

Noncritical State Transitions During Conditionally Automated Driving on German Freeways: Effects of Non-Driving Related Tasks on Takeover Time and Takeover Quality.

OBJECTIVE: This study aimed at investigating the driver's takeover performance when switching from working on different non-driving related tasks (NDRTs) while driving with a conditionally automated driving function (SAE L3), which was simulated by a Wizard of Oz vehicle, to manual vehicle control under naturalistic driving conditions. BACKGROUND: Conditionally automated driving systems, which are currently close to market introduction, require the user to stay fallback ready. As users will be allowed to engage in more complex NDRTs during the automated drive than when driving manually, the time needed to regain full manual control could likely be increased. METHOD: Thirty-four users engaged in different everyday NDRTs while driving automatically with a Wizard of Oz vehicle. After approximately either 5 min or 15 min of automated driving, users were requested to take back vehicle control in noncritical situations. The test drive took place in everyday traffic on German freeways in the metropolitan area of Stuttgart. RESULTS: Particularly tasks that required users to turn away from the central road scene or hold an object in their hands led to increased takeover times. Accordingly, increased variance in the driver's lane position was found shortly after the switch to manual control. However, the drivers rated the takeover situations to be mostly "harmless." CONCLUSION: Drivers managed to regain control over the vehicle safely, but they needed more time to prepare for the manual takeover when the NDRTs caused motoric workload. APPLICATION: The timings found in the study can be used to design comfortable and safe takeover concepts for automated vehicles.

From partial and high automation to manual driving: Relationship between non-driving related tasks, drowsiness and take-over performance.

BACKGROUND: Until the level of full vehicle automation is reached, users of vehicle automation systems will be required to take over manual control of the vehicle occasionally and stay fallback-ready to some extent during the drive. Both, drowsiness caused by inactivity and the engagement in distracting non-driving related tasks (NDRTs) such as entertainment or office work have been suggested to impair the driver's ability to safely handle these transitions of control. Thus, it is an open question whether engagement in NDRTs will impair or improve take-over performance. METHOD: In a motion-based driving simulator, 64 participants completed an automated drive that lasted either one or two hours using either a partially or highly automated driving system. In the partially automated driving condition, a warning was issued after several seconds when drivers took both hands off the steering wheel, while the highly automated driving system allowed hands-off driving permanently. Drivers were allowed to bring along their smartphones and to use them during the drive. They engaged in a wide variety of NDRTs such as reading or using social media. At the end of the session, drivers had to react to a sudden lead vehicle braking event. In the partial automation condition, there was no take-over request (TOR) to notify the drivers of the braking vehicle, while in the highly automated condition, the situation happened right after the drivers had deactivated the automation in response to a TOR. The lead time of the TOR was set at 8 s. Driver's level of drowsiness, workload (visual, mental and motoric) from carrying out the NDRT and motivational appeal of the NDRT right before the control transition were video-coded and used to predict the outcome of the braking event (i.e., reaction and system deactivation times, minimal Time-to-collision (TTC) and self-reported criticality) with a multiple regression approach. RESULTS: In the partial automation condition, reaction times to the braking vehicle and situation criticality as measured by the minimum TTC could be well predicted. Main predictors for increased reaction time were drowsiness and motivational appeal of the NDRT. However, visual and mental demand associated with NDRTs did decrease reaction time, suggesting that the NDRT helped the drivers to maintain alertness during the partially automated drive. Accordingly, drowsiness and motivational appeal of the NDRT increased situation criticality, while cognitive load due to the NDRT decreased it. In the highly automated condition, however, it was not possible to predict system deactivation time (in reaction to the TOR), brake reaction time to the braking vehicle and situation criticality by observed drowsiness and NDRT engagement. DISCUSSION: The results suggest a relationship between the driver's drowsiness and NDRT engagement in partial automation but not in highly automated driving. Several explanations for this finding are discussed. It could be possible that the lead time of 8 s might have given the drivers enough time to complete the driver state transition process from executing NDRTs to manual driving, putting them in a position to be able to cope with the driving event, while this was not possible in the partial automation condition. Methodological issues that might have led to a non-detection of an effect of drowsiness or NDRT engagement in the highly automated driving condition, such as the sample size and sensitivity of the observer ratings, are also discussed.

Expert-based controllability assessment of control transitions from automated to manual driving.

Up to a level of full vehicle automation, drivers will have to be available as a fallback level and take back manual control of the vehicle in case of system limits or failures. Before introducing automated vehicles to the consumer market, the controllability of these control transitions has to be demonstrated. This paper presents a novel procedure for an expert-based controllability assessment of control transitions from automated to manual driving. A standardized rating scheme is developed that allows trained raters to integrate different aspects of driving performance during control transitions (e.g., quality of lateral and longitudinal control, adequateness of signalling to other road users, etc.) into one global controllability measure based on video material of the driving situation. The method is adapted from an existing assessment procedure that has been successfully applied to assess the criticality of driving situations in manual driving conditions (e.g., assessment of substance-induced impairments, assessment of fitness-to-drive of novice drivers). This paper presents the rating procedure, including instructions of how to code relevant qualities of the drivers' performance with accompanying video-demonstrations, and material used for rater training. •A rating procedure for an expert-based controllability assessment of control transitions from automated to manual driving based on observation of video material was adapted from an existing method used in studies on manual driving.•The advantage of this method consists in an integration of different dimensions of driving performance (e.g., operational and tactical driving behaviour, criticality of the situation) into one global controllability measure.•The method allows an assessment and comparison of diverse take-over scenarios, detached from driver performance variables.•The accompanying video-based training material allows reproducible and reliable execution of the rating procedure.

Effect of different alcohol levels on take-over performance in conditionally automated driving.

Automated driving systems are getting pushed into the consumer market, with varying degrees of automation. Most often the driver's task will consist of being available as a fall-back level when the automation reaches its limits. These so-called take-over situations have attracted a great body of research, focusing on various human factors aspects (e.g., sleepiness) that could undermine the safety of control transitions between automated and manual driving. However, a major source of accidents in manual driving, alcohol consumption, has been a non-issue so far, although a false understanding of the driver's responsibility (i.e., being available as a fallback level) might promote driving under its influence. In this experiment, N = 36 drivers were exposed to different levels of blood alcohol concentrations (BACs: placebo vs. 0.05% vs. 0.08%) in a high fidelity driving simulator, and the effect on take-over time and quality was assessed. The results point out that a 0.08% BAC increases the time needed to re-engage in the driving task and impairs several aspects of longitudinal and lateral vehicle control, whereas 0.05% BAC did only go along with descriptive impairments in fewer parameters.

Driver compliance to take-over requests with different auditory outputs in conditional automation.

Conditionally automated driving (CAD) systems are expected to improve traffic safety. Whenever the CAD system exceeds its limit of operation, designers of the system need to ensure a safe and timely enough transition from automated to manual mode. An existing visual Human-Machine Interface (HMI) was supplemented by different auditory outputs. The present work compares the effects of different auditory outputs in form of (1) a generic warning tone and (2) additional semantic speech output on driver behavior for the announcement of an upcoming take-over request (TOR). We expect the information carried by means of speech output to lead to faster reactions and better subjective evaluations by the drivers compared to generic auditory output. To test this assumption, N=17 drivers completed two simulator drives, once with a generic warning tone ('Generic') and once with additional speech output ('Speech+generic'), while they were working on a non-driving related task (NDRT; i.e., reading a magazine). Each drive incorporated one transition from automated to manual mode when yellow secondary lanes emerged. Different reaction time measures, relevant for the take-over process, were assessed. Furthermore, drivers evaluated the complete HMI regarding usefulness, ease of use and perceived visual workload just after experiencing the take-over. They gave comparative ratings on usability and acceptance at the end of the experiment. Results revealed that reaction times, reflecting information processing time (i.e., hands on the steering wheel, termination of NDRT), were shorter for 'Speech+generic' compared to 'Generic' while reaction time, reflecting allocation of attention (i.e., first glance ahead), did not show this difference. Subjective ratings were in favor of the system with additional speech output.

Driving performance at lateral system limits during partially automated driving.

This study investigated driver performance during system limits of partially automated driving. Using a motion-based driving simulator, drivers encountered different situations in which a partially automated vehicle could no longer safely keep the lateral guidance. Drivers were distracted by a non-driving related task on a touch display or driving without an additional secondary task. While driving in partially automated mode drivers could either take their hands off the steering wheel for only a short period of time (10s, so-called 'Hands-on' variant) or for an extended period of time (120s, so-called 'Hands-off' variant). When the system limit was reached (e.g., when entering a work zone with temporary lines), the lateral vehicle control by the automation was suddenly discontinued and a take-over request was issued to the drivers. Regardless of the hands-off interval and the availability of a secondary task, all drivers managed the transition to manual driving safely. No lane exceedances were observed and the situations were rated as 'harmless' by the drivers. The lack of difference between the hands-off intervals can be partly attributed to the fact that most of the drivers kept contact to the steering wheel, even in the hands-off condition. Although all drivers were able to control the system limits, most of them could not explain why exactly the take-over request was issued. The average helpfulness of the take-over request was rated on an intermediate level. Consequently, providing drivers with information about the reason for a system limit can be recommended.

Evaluating distraction of in-vehicle information systems while driving by predicting total eyes-off-road times with keystroke level modeling.

Increasingly complex in-vehicle information systems (IVIS) have become available in the automotive vehicle interior. To ensure usability and safety of use while driving, the distraction potential of system-associated tasks is most often analyzed during the development process, either by employing empirical or analytical methods, with both families of methods offering certain advantages and disadvantages. The present paper introduces a method that combines the predictive precision of empirical methods with the economic advantages of analytical methods. Keystroke level modeling (KLM) was extended to a task-dependent modeling procedure for total eyes-off-road times (TEORT) resulting from system use while driving and demonstrated by conducting two subsequent simulator studies. The first study involved the operation of an IVIS by N = 18 participants. The results suggest a good model fit (R(2)Adj. = 0.67) for predicting the TEORT, relying on regressors from KLM and participant age. Using the parameter estimates from study 1, the predictive validity of the model was successfully tested during a second study with N = 14 participants using a version of the IVIS prototype with a revised design and task structure (rPred.-Obs. = 0.58). Possible applications and shortcomings of the approach are discussed.

Cooperative warning systems: The impact of false and unnecessary alarms on drivers' compliance.

Cooperative warning systems have a great potential to prevent traffic accidents. However, because of their predictive nature, they might also go along with an increased frequency of incorrect alarms that could limit their effectiveness. To better understand the consequences associated with incorrect alarms, a driving simulator study with N=80 drivers was conducted to investigate how situational context and warning urgency jointly influence drivers' compliance with an unreliable advisory warning system (AWS). The participants encountered several critical urban driving situations and were either assisted by a 100% reliable AWS, a 60% reliable AWS that generated false alarms (without obvious reason) or a 60% reliable AWS that generated unnecessary alarms (with plausible reason). A baseline drive without any assistance was also introduced to the study. The warnings were presented either only visually or visual-auditory. In line with previous research, drivers' compliance and effectiveness of the AWS was reduced by false alarms but not by unnecessary alarms. However, this so-called cry wolf effect (Breznitz, 1984) was only found in the visual-auditory condition, whereas there was no effect of warning reliability in the condition with visual AWS. Furthermore, false but not unnecessary alarms caused the participants to rate the AWS less favourably during a follow-up interview. In spite of these negative effects of false alarms, a reduction in the frequency of safety-critical events (SCEs) and an earlier braking onset were evident in all assisted drives compared with that of non-assisted driving, even when the AWS was unreliable. The results may thus lower concerns about the negative consequences of warning drivers unnecessarily about upcoming traffic conflicts if the reasons of these alarms are comprehensible. From a perspective of designing AWS, we recommend to use less urgent warnings to prevent the cry wolf effect.