Modeling Brain Dynamics During Virtual Reality-Based Emergency Response Learning Under Stress.

BACKGROUND: Stress affects learning during training, and virtual reality (VR) based training systems that manipulate stress can improve retention and retrieval performance for firefighters. Brain imaging using functional Near Infrared Spectroscopy (fNIRS) can facilitate development of VR-based adaptive training systems that can continuously assess the trainee's states of learning and cognition. OBJECTIVE: The aim of this study was to model the neural dynamics associated with learning and retrieval under stress in a VR-based emergency response training exercise. METHODS: Forty firefighters underwent an emergency shutdown training in VR and were randomly assigned to either a control or a stress group. The stress group experienced stressors including smoke, fire, and explosions during the familiarization and training phase. Both groups underwent a stress memory retrieval and no-stress memory retrieval condition. Participant's performance scores, fNIRS-based neural activity, and functional connectivity between the prefrontal cortex (PFC) and motor regions were obtained for the training and retrieval phases. RESULTS: The performance scores indicate that the rate of learning was slower in the stress group compared to the control group, but both groups performed similarly during each retrieval condition. Compared to the control group, the stress group exhibited suppressed PFC activation. However, they showed stronger connectivity within the PFC regions during the training and between PFC and motor regions during the retrieval phases. DISCUSSION: While stress impaired performance during training, adoption of stress-adaptive neural strategies (i.e., stronger brain connectivity) were associated with comparable performance between the stress and the control groups during the retrieval phase.

Physiological and perceptual consequences of trust in collaborative robots: An empirical investigation of human and robot factors.

Measuring trust is an important element of effective human-robot collaborations (HRCs). It has largely relied on subjective responses and thus cannot be readily used for adapting robots in shared operations, particularly in shared-space manufacturing applications. Additionally, whether trust in such HRCs differ under altered operator cognitive states or with sex remains unknown. This study examined the impacts of operator cognitive fatigue, robot reliability, and operator sex on trust symptoms in collaborative robots through both objective measures (i.e., performance, heart rate variability) and subjective measures (i.e., surveys). Male and female participants were recruited to perform a metal surface polishing task in partnership with a collaborative robot (UR10), in which they underwent reliability conditions (reliable, unreliable) and cognitive fatigue conditions (fatigued, not fatigued). As compared to the reliable conditions, unreliable robot manipulations resulted in perceived trust, an increase in both sympathetic and parasympathetic activity, and operator-induced reduction in task efficiency and accuracy but not precision. Cognitive fatigue was shown to correlate with higher fatigue scores and reduced task efficiency, more severely impacting females. The results highlight key interplays between operator states of fatigue, sex, and robot reliability on both subjective and objective responses of trust. These findings provide a strong foundation for future investigations on better understanding the relationship between human factors and trust in HRC as well as aid in developing more diagnostic and deployable measures of trust.

Trust in Shared-Space Collaborative Robots: Shedding Light on the Human Brain.

BACKGROUND: Industry 4.0 is currently underway allowing for improved manufacturing processes that leverage the collective advantages of human and robot agents. Consideration of trust can improve the quality and safety in such shared-space human-robot collaboration environments. OBJECTIVE: The use of physiological response to monitor and understand trust is currently limited due to a lack of knowledge on physiological indicators of trust. This study examines neural responses to trust within a shared-workcell human-robot collaboration task as well as discusses the use of granular and multimodal perspectives to study trust. METHODS: Sixteen sex-balanced participants completed a surface finishing task in collaboration with a UR10 collaborative robot. All participants underwent robot reliability conditions and robot assistance level conditions. Brain activation and connectivity using functional near infrared spectroscopy, subjective responses, and performance were measured throughout the study. RESULTS: Significantly, increased neural activation was observed in response to faulty robot behavior within the medial and right dorsolateral prefrontal cortex (PFC). A similar trend was observed for the anterior PFC, primary motor cortex, and primary visual cortex. Faulty robot behavior also resulted in reduced functional connectivity strengths throughout the brain. DISCUSSION: These findings implicate regions in the prefrontal cortex along with specific connectivity patterns as signifiers of distrusting conditions. The neural response may be indicative of how trust is influenced, measured, and manifested for human-robot collaboration that requires active teaming. APPLICATION: Neuroergonomic response metrics can reveal new perspectives on trust in automation that subjective responses alone are not able to provide.

Human Factors Considerations and Metrics in Shared Space Human-Robot Collaboration: A Systematic Review.

The degree of successful human-robot collaboration is dependent on the joint consideration of robot factors (RF) and human factors (HF). Depending on the state of the operator, a change in a robot factor, such as the behavior or level of autonomy, can be perceived differently and affect how the operator chooses to interact with and utilize the robot. This interaction can affect system performance and safety in dynamic ways. The theory of human factors in human-automation interaction has long been studied; however, the formal investigation of these HFs in shared space human-robot collaboration (HRC) and the potential interactive effects between covariate HFs (HF-HF) and HF-RF in shared space collaborative robotics requires additional investigation. Furthermore, methodological applications to measure or manipulate these factors can provide insights into contextual effects and potential for improved measurement techniques. As such, a systematic literature review was performed to evaluate the most frequently addressed operator HF states in shared space HRC, the methods used to quantify these states, and the implications of the states on HRC. The three most frequently measured states are: trust, cognitive workload, and anxiety, with subjective questionnaires universally the most common method to quantify operator states, excluding fatigue where electromyography is more common. Furthermore, the majority of included studies evaluate the effect of manipulating RFs on HFs, but few explain the effect of the HFs on system attributes or performance. For those that provided this information, HFs have been shown to impact system efficiency and response time, collaborative performance and quality of work, and operator utilization strategy.

Neural Correlates of Trust in Automation: Considerations and Generalizability Between Technology Domains.

Investigations into physiological or neurological correlates of trust has increased in popularity due to the need for a continuous measure of trust, including for trust-sensitive or adaptive systems, measurements of trustworthiness or pain points of technology, or for human-in-the-loop cyber intrusion detection. Understanding the limitations and generalizability of the physiological responses between technology domains is important as the usefulness and relevance of results is impacted by fundamental characteristics of the technology domains, corresponding use cases, and socially acceptable behaviors of the technologies. While investigations into the neural correlates of trust in automation has grown in popularity, there is limited understanding of the neural correlates of trust, where the vast majority of current investigations are in cyber or decision aid technologies. Thus, the relevance of these correlates as a deployable measure for other domains and the robustness of the measures to varying use cases is unknown. As such, this manuscript discusses the current-state-of-knowledge in trust perceptions, factors that influence trust, and corresponding neural correlates of trust as generalizable between domains.