Corrigendum: Opportunities for human factors in machine learning.

[This corrects the article DOI: 10.3389/frai.2023.1130190.].

Opportunities for human factors in machine learning.

Introduction: The field of machine learning and its subfield of deep learning have grown rapidly in recent years. With the speed of advancement, it is nearly impossible for data scientists to maintain expert knowledge of cutting-edge techniques. This study applies human factors methods to the field of machine learning to address these difficulties. Methods: Using semi-structured interviews with data scientists at a National Laboratory, we sought to understand the process used when working with machine learning models, the challenges encountered, and the ways that human factors might contribute to addressing those challenges. Results: Results of the interviews were analyzed to create a generalization of the process of working with machine learning models. Issues encountered during each process step are described. Discussion: Recommendations and areas for collaboration between data scientists and human factors experts are provided, with the goal of creating better tools, knowledge, and guidance for machine learning scientists.

A Methodology for Evaluating Operator Usage of Machine Learning Recommendations for Power Grid Contingency Analysis.

This work presents the application of a methodology to measure domain expert trust and workload, elicit feedback, and understand the technological usability and impact when a machine learning assistant is introduced into contingency analysis for real-time power grid simulation. The goal of this framework is to rapidly collect and analyze a broad variety of human factors data in order to accelerate the development and evaluation loop for deploying machine learning applications. We describe our methodology and analysis, and we discuss insights gained from a pilot participant about the current usability state of an early technology readiness level (TRL) artificial neural network (ANN) recommender.

Adaptive Cognitive Mechanisms to Maintain Calibrated Trust and Reliance in Automation.

Trust calibration for a human-machine team is the process by which a human adjusts their expectations of the automation's reliability and trustworthiness; adaptive support for trust calibration is needed to engender appropriate reliance on automation. Herein, we leverage an instance-based learning ACT-R cognitive model of decisions to obtain and rely on an automated assistant for visual search in a UAV interface. This cognitive model matches well with the human predictive power statistics measuring reliance decisions; we obtain from the model an internal estimate of automation reliability that mirrors human subjective ratings. The model is able to predict the effect of various potential disruptions, such as environmental changes or particular classes of adversarial intrusions on human trust in automation. Finally, we consider the use of model predictions to improve automation transparency that account for human cognitive biases in order to optimize the bidirectional interaction between human and machine through supporting trust calibration. The implications of our findings for the design of reliable and trustworthy automation are discussed.

The mixed blessings of smart infusion devices and health care IT.

From July to October 2009, a team of human factors researchers evaluated the use of a commercially available infusion device among nurses at a tertiary care hospital in the Midwest. The study's purpose was to determine the factors that may influence the adoption and "best practice" use of smart infusion devices by identifying the human, technological, environmental, and/or organizational factors and to describe how they support or impede safe practices. The study's aim was to show how technology and individual and team behavior influence each other, as well as care performance and outcomes. Research team members shadowed nursing personnel as they performed routine care activities, and conducted cognitive task analysis interviews with nurses, an engineer, and a pharmacist. They identified key themes, and then made several systematic passes through the data to identify all instances of each theme and to collect examples and illustrative quotes. Although staff members were positive in their comments about the smart pump, observations and interviews revealed discrepancies between prescriptions and infusions, and "workarounds" to cope with the mismatch between interface design and actual care requirements. Despite "smart pump" capabilities, situations continue such as the need for clinicians to perform calculations in order to deliver medications. These workarounds, which make them and patients vulnerable to adverse outcomes, confirm prior published research by Cook, Nemeth, Nunnally, Hollnagel, and Woods. The team provided recommendations based on findings for training and interface design.

The role of alarm signal duration as a cue for alarm validity.

Researchers have isolated several variables that moderate the degrading effects of alarm mistrust on alarm reactions. We examined how alarm duration influences reactions to alarms of varying true alarm rates. In Experiment one, 45 psychology students performed a complex psychomotor task while reacting to an alarm system generating short- and long-duration signals. We predicted that participants would consider long-duration alarms more valid and would respond more to them despite the true alarm rate. Results supported both expectations. In addition to these findings, participants believed that true alarm rate influenced their response decisions more than duration even though true alarm rate did not affect actual response frequency. In Experiment two, 40 students reacted to short- and long-duration alarms originating from unique systems. Results showed some participants relied on duration, whereas others used true alarm rate, responded extremely, or combined strategies. Overall, results suggest signal duration is an important influence, but that increased task complexity may lead operators to adopt other reaction strategies.