Multilevel Confirmatory Factor Analysis Reveals Two Distinct Human-Automation Trust Constructs.

OBJECTIVE: This work examined the relationship of the constructs measured by the trust scales developed by Chancey et al. (2017) and Jian et al. (2000) using a multilevel confirmatory factor analysis (CFA). BACKGROUND: Modern theories of automation trust have been proposed based on data collected using trust scales. Chancey et al. (2017) adapted Madsen and Gregor's (2000) trust scale to align with Lee and See's (2004) human-automation trust framework. In contrast, Jian et al. (2000) developed a scale empirically with trust and distrust as factors. However, it remains unclear whether these two scales measure the same construct. METHOD: We analyzed data collected from previous experiments to investigate the relationship between the two trust scales using a multilevel CFA. RESULTS: Data provided evidence that Jian et al. (2000) and Chancey et al. (2017) automation trust scales are only weakly related. Trust and distrust are found to be distinct factors in Jian et al.'s (2000) scale, whereas performance, process, and purpose are distinct factors in Chancey et al.'s (2017) trust scale. CONCLUSION: The analysis suggested that the two scales purporting to measure human-automation trust are only weakly related. APPLICATION: Trust researchers and automation designers may consider using Chancey et al. (2017) and Jian et al. (2000) scales to capture different characteristics of human-automation trust.

Automation trust and attention allocation in multitasking workspace.

Previous research suggests that operators with high workload can distrust and then poorly monitor automation, which has been generally inferred from automation dependence behaviors. To test automation monitoring more directly, the current study measured operators' visual attention allocation, workload, and trust toward imperfect automation in a dynamic multitasking environment. Participants concurrently performed a manual tracking task with two levels of difficulty and a system monitoring task assisted by an unreliable signaling system. Eye movement data indicate that operators allocate less visual attention to monitor automation when the tracking task is more difficult. Participants reported reduced levels of trust toward the signaling system when the tracking task demanded more focused visual attention. Analyses revealed that trust mediated the relationship between the load of the tracking task and attention allocation in Experiment 1, an effect that was not replicated in Experiment 2. Results imply a complex process underlying task load, visual attention allocation, and automation trust during multitasking. Automation designers should consider operators' task load in multitasking workspaces to avoid reduced automation monitoring and distrust toward imperfect signaling systems.

Trust and the Compliance-Reliance Paradigm: The Effects of Risk, Error Bias, and Reliability on Trust and Dependence.

OBJECTIVE: This study provides a theoretical link between trust and the compliance-reliance paradigm. We propose that for trust mediation to occur, the operator must be presented with a salient choice, and there must be an element of risk for dependence. BACKGROUND: Research suggests that false alarms and misses affect dependence via two independent processes, hypothesized as trust in signals and trust in nonsignals. These two trust types manifest in categorically different behaviors: compliance and reliance. METHOD: Eighty-eight participants completed a primary flight task and a secondary signaling system task. Participants evaluated their trust according to the informational bases of trust: performance, process, and purpose. Participants were in a high- or low-risk group. Signaling systems varied by reliability (90%, 60%) within subjects and error bias (false alarm prone, miss prone) between subjects. RESULTS: False-alarm rate affected compliance but not reliance. Miss rate affected reliance but not compliance. Mediation analyses indicated that trust mediated the relationship between false-alarm rate and compliance. Bayesian mediation analyses favored evidence indicating trust did not mediate miss rate and reliance. Conditional indirect effects indicated that factors of trust mediated the relationship between false-alarm rate and compliance (i.e., purpose) and reliance (i.e., process) but only in the high-risk group. CONCLUSION: The compliance-reliance paradigm is not the reflection of two types of trust. APPLICATION: This research could be used to update training and design recommendations that are based upon the assumption that trust causes operator responses regardless of error bias.

The Role of Trust as a Mediator Between System Characteristics and Response Behaviors.

OBJECTIVE: The purpose of the current work was to clarify how subjective trust determines response behavior when interacting with a signaling system. BACKGROUND: In multiple theoretical frameworks, trust is acknowledged as a prime mediator between system error characteristics and automation dependence. Some researchers have operationally defined trust as the behavior exhibited. Other researchers have suggested that although trust may guide operator responses, trust does not completely determine the behavior. METHOD: Forty-four participants interacted with a primary flight simulation task and a secondary signaling system task. The signaling system varied in reliability (90%, 60%) and error bias (false alarm, miss prone). Trust was measured halfway through the experimental session to address the criterion of temporal precedence in determining the effect of trust on behavior. RESULTS: Analyses indicated that trust partially mediated the relationship between reliability and agreement rate. Trust did not mediate the relationship between reliability and reaction time. Trust also did not mediate the relationships between error bias and reaction time or agreement rate. Analyses of variance generally supported specific behavioral and trust hypotheses, indicating that the paradigm employed produced similar effects on response behaviors and subjective estimates of trust observed in other studies. CONCLUSION: These results indicate that strong assumptions of trust acting as the prime mediator between system error characteristics and response behaviors should be viewed with caution. APPLICATION: Practitioners should consider assessing factors other than trust to determine potential operator response behaviors, which may be more predictive.

An investigation of training strategies to improve alarm reactions.

Researchers have suggested that operator training may improve operator reactions; however, researchers have not documented this for alarm reactions. The goal of this research was to train participants to react to alarms using sensor activity patterns. In Experiment 1, 80 undergraduates monitored a simulated security screen while completing a primary word search task. They received spatial, temporal, single sensor, or no training to respond to alarms of differing reliability levels. Analyses revealed more appropriate and quicker reactions when participants were trained and when the alarms were reliable. In Experiment 2, 56 participants practiced time estimation by simple repetition, performance feedback, or performance feedback and temporal subdivision. They then reacted to alarms based on elapsed time between sensor activity and alarm onset. Surprisingly, results indicated that participants did not benefit differentially from temporal interval training, focusing instead on advertised system reliability. Researchers should replicate these findings with realistic tasks and real-world complex task operators.