Prediction of cognitive conflict during unexpected robot behavior under different mental workload conditions in a physical human-robot collaboration.

Objective. Brain-computer interface (BCI) technology is poised to play a prominent role in modern work environments, especially a collaborative environment where humans and machines work in close proximity, often with physical contact. In a physical human robot collaboration (pHRC), the robot performs complex motion sequences. Any unexpected robot behavior or faulty interaction might raise safety concerns. Error-related potentials, naturally generated by the brain when a human partner perceives an error, have been extensively employed in BCI as implicit human feedback to adapt robot behavior to facilitate a safe and intuitive interaction. However, the integration of BCI technology with error-related potential for robot control demands failure-free integration of highly uncertain electroencephalography (EEG) signals, particularly influenced by the physical and cognitive state of the user. As a higher workload on the user compromises their access to cognitive resources needed for error awareness, it is crucial to study how mental workload variations impact the error awareness as it might raise safety concerns in pHRC. In this study, we aim to study how cognitive workload affects the error awareness of a human user engaged in a pHRC.Approach. We designed a blasting task with an abrasive industrial robot and manipulated the mental workload with a secondary arithmetic task of varying difficulty. EEG data, perceived workload, task and physical performance were recorded from 24 participants moving the robot arm. The error condition was achieved by the unexpected stopping of the robot in 33% of trials.Main results. We observed a diminished amplitude for the prediction error negativity (PEN) and error positivity (Pe), indicating reduced error awareness with increasing mental workload. We further observed an increased frontal theta power and increasing trend in the central alpha and central beta power after the unexpected robot stopping compared to when the robot stopped correctly at the target. We also demonstrate that a popular convolution neural network model, EEGNet, could predict the amplitudes of PEN and Pe from the EEG data prior to the error.Significance. This prediction model could be instrumental in developing an online prediction model that could forewarn the system and operators of the diminished error awareness of the user, alluding to a potential safety breach in error-related potential-based BCI system for pHRC. Therefore, our work paves the way for embracing BCI technology in pHRC to optimally adapt the robot behavior for personalized user experience using real-time brain activity, enriching the quality of the interaction.

Real-time Estimation of the Strength Capacity of the Upper Limb for Physical Human-Robot Collaboration.

In physical Human-Robot Collaboration (pHRC), having an estimate of the operator's strength capacity can help implement control strategies. Currently, the trend is to integrate devices that can measure physiological signals. This is not always a viable option, especially for highly dynamic tasks. For pHRC tasks, the physical interaction point usually occurs at the operator's hand. Therefore, a musculo-skeletal model was used to have a real-time estimation of the strength capacity of the operator's upper limb. First, the model has been simplified to reduce the complexity of the problem. The model was used to obtain offline estimations of the strength capacity, that were then curve-fitted to enable real-time estimation. An experiment was carried out to compare the results of the approximated model with human data. Results suggest that this method for estimating the strength capacity of the upper limb is a viable solution for real-time applications.

Upper limb strength estimation of physically impaired persons using a musculoskeletal model: A sensitivity analysis.

Sensitivity of upper limb strength calculated from a musculoskeletal model was analyzed, with focus on how the sensitivity is affected when the model is adapted to represent a person with physical impairment. Sensitivity was calculated with respect to four muscle-tendon parameters: muscle peak isometric force, muscle optimal length, muscle pennation, and tendon slack length. Results obtained from a musculoskeletal model of average strength showed highest sensitivity to tendon slack length, followed by muscle optimal length and peak isometric force, which is consistent with existing studies. Muscle pennation angle was relatively insensitive. The analysis was repeated after adapting the musculoskeletal model to represent persons with varying severities of physical impairment. Results showed that utilizing the weakened model significantly increased the sensitivity of the calculated strength at the hand, with parameters previously insensitive becoming highly sensitive. This increased sensitivity presents a significant challenge in applications utilizing musculoskeletal models to represent impaired individuals.

Experimental evaluation of a model-based assistance-as-needed paradigm using an assistive robot.

In robotic rehabilitation a promising paradigm is assistance-as-needed. This is because it promotes patient active participation which is essential for neuro-rehabilitation. A model-based assistance-as-needed paradigm has been developed which utilizes a musculoskeletal model representing the subject to calculate their assistance needs. In this paper we experimentally evaluate this model-based paradigm to control an assistive robot and provide a subject with assistance-as-needed at the muscular level. A subject with impairments defined in specific muscle groups performs a number of upper limb tasks, whilst receiving assistance from a robotic exoskeleton. The paradigm is evaluated on its ability to provide assistance only as the subject needs, depending on the tasks being performed and the impairments defined. Results show that the model-based assistance-as-needed paradigm was relatively successful in providing assistance when it was needed.

Admittance control scheme for implementing model-based assistance-as-needed on a robot.

A model-based assistance-as-needed paradigm has been developed to govern the assistance provided by an assistive robot to its operator. This paradigm has advantages over existing methods of providing assistance-as-needed for applications such as robotic rehabilitation. However, implementation of the model-based paradigm requires a control scheme to be developed which controls the robot so as to provide the assistance calculated by the model-based paradigm to its operator. In this paper an admittance control scheme for providing model-based assistance-as-needed is presented. It is developed considering its suitability for human-robot interaction, and its role within the model-based assistance-as-needed framework. Results from the control implemented on an example robot showed it is capable of providing the operator with the desired level of assistance as governed by the model-based paradigm. This is an essential requirement for the paradigm to be capable of providing efficacious assistance-as-needed in applications such as robotic rehabilitation.

Estimating physical assistance need using a musculoskeletal model.

Technologies that provide physical assistance during tasks are often required to provide assistance specific to the task and person performing it. An example is robotic rehabilitation in which the assistance-as-needed (AAN) paradigm aims to provide operators with the minimum assistance required to perform the task. Current approaches use empirical performance-based methods which require repeated observation of the specific task before an estimate of the needed assistance can be determined. In this paper, we present a new approach utilizing a musculoskeletal model (MM) of the upper limb to estimate the operator's assistance needs with respect to physical tasks. With capabilities of the operator defined at the muscular level of the MM, an optimization model is used to estimate the operator's strength capability. Strength required to perform a task is calculated using a task model. The difference or gap between the operator's strength capability and the strength required to execute a task forms the basis for the new AAN paradigm. We show how this approach estimates the effects of limb pose, load direction, and muscle impairments on a person's ability to perform tasks.

A task description model for robotic rehabilitation.

The desire to produce robots to aid in physical neurorehabilitation has led to the control paradigm Assistance-As-Needed. This paradigm aims to assist patients in performing physical rehabilitation tasks whilst providing the least amount of assistance required, maximizing the patient's effort which is essential for recovery. Ideally the provided assistance equals the gap between the capability required to perform the task and the patient's available capability. Current implementations derive a measure of this gap by critiquing task performance based on some criteria. This paper presents a task description model for tasks performed by a patient's limb, allowing physical requirements to be calculated. Applied to two upper limb tasks typical of rehabilitation and daily activities, the effect of task variations on the task's physical requirements are observed. It is proposed that using the task description model to compensate for changing task requirements will allow better support by providing assistance closer to the true needs of the patient.

Towards using musculoskeletal models for intelligent control of physically assistive robots.

With the increasing number of robots being developed to physically assist humans in tasks such as rehabilitation and assistive living, more intelligent and personalized control systems are desired. In this paper we propose the use of a musculoskeletal model to estimate the strength of the user, from which information can be utilized to improve control schemes in which robots physically assist humans. An optimization model is developed utilizing a musculoskeletal model to estimate human strength in a specified dynamic state. Results of this optimization as well as methods of using it to observe muscle-based weaknesses in task space are presented. Lastly potential methods and problems in incorporating this model into a robot control system are discussed.