Flow in human-robot collaboration-multimodal analysis and perceived challenge detection in industrial scenarios.

Introduction: Flow state, the optimal experience resulting from the equilibrium between perceived challenge and skill level, has been extensively studied in various domains. However, its occurrence in industrial settings has remained relatively unexplored. Notably, the literature predominantly focuses on Flow within mentally demanding tasks, which differ significantly from industrial tasks. Consequently, our understanding of emotional and physiological responses to varying challenge levels, specifically in the context of industry-like tasks, remains limited. Methods: To bridge this gap, we investigate how facial emotion estimation (valence, arousal) and Heart Rate Variability (HRV) features vary with the perceived challenge levels during industrial assembly tasks. Our study involves an assembly scenario that simulates an industrial human-robot collaboration task with three distinct challenge levels. As part of our study, we collected video, electrocardiogram (ECG), and NASA-TLX questionnaire data from 37 participants. Results: Our results demonstrate a significant difference in mean arousal and heart rate between the low-challenge (Boredom) condition and the other conditions. We also found a noticeable trend-level difference in mean heart rate between the adaptive (Flow) and high-challenge (Anxiety) conditions. Similar differences were also observed in a few other temporal HRV features like Mean NN and Triangular index. Considering the characteristics of typical industrial assembly tasks, we aim to facilitate Flow by detecting and balancing the perceived challenge levels. Leveraging our analysis results, we developed an HRV-based machine learning model for discerning perceived challenge levels, distinguishing between low and higher-challenge conditions. Discussion: This work deepens our understanding of emotional and physiological responses to perceived challenge levels in industrial contexts and provides valuable insights for the design of adaptive work environments.

Azure Kinect performance evaluation for human motion and upper limb biomechanical analysis.

Human motion tracking is a valuable task for many medical applications. Marker-based optoelectronic systems are considered the gold standard in human motion tracking. However, their use is not always feasible in clinics and industrial environments. On the other hand, marker-less sensors became valuable tools, as they are inexpensive, noninvasive and easy to use. However, their accuracy can depend on many factors including sensor positioning, light conditions and body occlusions. In this study, following previous works on the feasibility of marker-less systems for human motion monitoring, we investigate the performance of the Microsoft Azure Kinect sensor in computing kinematic and dynamic measurements of static postures and dynamic movements. According to our knowledge, it is the first time that this sensor is compared with a Vicon marker-based system to assess the best camera positioning while observing the upper body part movements of people performing several tasks. Twenty-five healthy volunteers were monitored to evaluate the effects of the several testing conditions, including the Azure Kinect positions, the light conditions, and lower limbs occlusions, on the tracking accuracy of kinematic, dynamic, and motor control parameters. From the statistical analysis of the performed measurements, the camera in the frontal position was the most reliable, the lighting conditions had almost no effects on the tracking accuracy, while the lower limbs occlusion worsened the accuracy of the upper limbs. The assessment of human static postures and dynamic movements based on experimental data proves the feasibility of applying the Azure Kinect to the biomechanical monitoring of human motion in several fields.

Behavioral patterns in robotic collaborative assembly: comparing neurotypical and Autism Spectrum Disorder participants.

Introduction: In Industry 4.0, collaborative tasks often involve operators working with collaborative robots (cobots) in shared workspaces. Many aspects of the operator's well-being within this environment still need in-depth research. Moreover, these aspects are expected to differ between neurotypical (NT) and Autism Spectrum Disorder (ASD) operators. Methods: This study examines behavioral patterns in 16 participants (eight neurotypical, eight with high-functioning ASD) during an assembly task in an industry-like lab-based robotic collaborative cell, enabling the detection of potential risks to their well-being during industrial human-robot collaboration. Each participant worked on the task for five consecutive days, 3.5 h per day. During these sessions, six video clips of 10 min each were recorded for each participant. The videos were used to extract quantitative behavioral data using the NOVA annotation tool and analyzed qualitatively using an ad-hoc observational grid. Also, during the work sessions, the researchers took unstructured notes of the observed behaviors that were analyzed qualitatively. Results: The two groups differ mainly regarding behavior (e.g., prioritizing the robot partner, gaze patterns, facial expressions, multi-tasking, and personal space), adaptation to the task over time, and the resulting overall performance. Discussion: This result confirms that NT and ASD participants in a collaborative shared workspace have different needs and that the working experience should be tailored depending on the end-user's characteristics. The findings of this study represent a starting point for further efforts to promote well-being in the workplace. To the best of our knowledge, this is the first work comparing NT and ASD participants in a collaborative industrial scenario.

Biomechanical Assessments of the Upper Limb for Determining Fatigue, Strain and Effort from the Laboratory to the Industrial Working Place: A Systematic Review.

Recent human-centered developments in the industrial field (Industry 5.0) lead companies and stakeholders to ensure the wellbeing of their workers with assessments of upper limb performance in the workplace, with the aim of reducing work-related diseases and improving awareness of the physical status of workers, by assessing motor performance, fatigue, strain and effort. Such approaches are usually developed in laboratories and only at times they are translated to on-field applications; few studies summarized common practices for the assessments. Therefore, our aim is to review the current state-of-the-art approaches used for the assessment of fatigue, strain and effort in working scenarios and to analyze in detail the differences between studies that take place in the laboratory and in the workplace, in order to give insights on future trends and directions. A systematic review of the studies aimed at evaluating the motor performance, fatigue, strain and effort of the upper limb targeting working scenarios is presented. A total of 1375 articles were found in scientific databases and 288 were analyzed. About half of the scientific articles are focused on laboratory pilot studies investigating effort and fatigue in laboratories, while the other half are set in working places. Our results showed that assessing upper limb biomechanics is quite common in the field, but it is mostly performed with instrumental assessments in laboratory studies, while questionnaires and scales are preferred in working places. Future directions may be oriented towards multi-domain approaches able to exploit the potential of combined analyses, exploitation of instrumental approaches in workplace, targeting a wider range of people and implementing more structured trials to translate pilot studies to real practice.

The effects of robotic assistance on upper limb spatial muscle synergies in healthy people during planar upper-limb training.

BACKGROUND: Robotic rehabilitation is a commonly adopted technique used to restore motor functionality of neurological patients. However, despite promising results were achieved, the effects of human-robot interaction on human motor control and the recovery mechanisms induced with robot assistance can be further investigated even on healthy subjects before translating to clinical practice. In this study, we adopt a standard paradigm for upper-limb rehabilitation (a planar device with assistive control) with linear and challenging curvilinear trajectories to investigate the effect of the assistance in human-robot interaction in healthy people. METHODS: Ten healthy subjects were instructed to perform a large set of radial and curvilinear movements in two interaction modes: 1) free movement (subjects hold the robot handle with no assistance) and 2) assisted movement (with a force tunnel assistance paradigm). Kinematics and EMGs from representative upper-limb muscles were recorded to extract phasic muscle synergies. The free and assisted interaction modes were compared assessing the level of assistance, error, and muscle synergy comparison between the two interaction modes. RESULTS: It was found that in free movement error magnitude is higher than with assistance, proving that task complexity required assistance also on healthy controls. Moreover, curvilinear tasks require more assistance than standard radial paths and error is higher. Interestingly, while assistance improved task performance, we found only a slight modification of phasic synergies when comparing assisted and free movement. CONCLUSIONS: We found that on healthy people, the effect of assistance was significant on task performance, but limited on muscle synergies. The findings of this study can find applications for assessing human-robot interaction and to design training to maximize motor recovery.