Flow in human-robot collaboration-multimodal analysis and perceived challenge detection in industrial scenarios.

Introduction: Flow state, the optimal experience resulting from the equilibrium between perceived challenge and skill level, has been extensively studied in various domains. However, its occurrence in industrial settings has remained relatively unexplored. Notably, the literature predominantly focuses on Flow within mentally demanding tasks, which differ significantly from industrial tasks. Consequently, our understanding of emotional and physiological responses to varying challenge levels, specifically in the context of industry-like tasks, remains limited. Methods: To bridge this gap, we investigate how facial emotion estimation (valence, arousal) and Heart Rate Variability (HRV) features vary with the perceived challenge levels during industrial assembly tasks. Our study involves an assembly scenario that simulates an industrial human-robot collaboration task with three distinct challenge levels. As part of our study, we collected video, electrocardiogram (ECG), and NASA-TLX questionnaire data from 37 participants. Results: Our results demonstrate a significant difference in mean arousal and heart rate between the low-challenge (Boredom) condition and the other conditions. We also found a noticeable trend-level difference in mean heart rate between the adaptive (Flow) and high-challenge (Anxiety) conditions. Similar differences were also observed in a few other temporal HRV features like Mean NN and Triangular index. Considering the characteristics of typical industrial assembly tasks, we aim to facilitate Flow by detecting and balancing the perceived challenge levels. Leveraging our analysis results, we developed an HRV-based machine learning model for discerning perceived challenge levels, distinguishing between low and higher-challenge conditions. Discussion: This work deepens our understanding of emotional and physiological responses to perceived challenge levels in industrial contexts and provides valuable insights for the design of adaptive work environments.

Azure Kinect performance evaluation for human motion and upper limb biomechanical analysis.

Human motion tracking is a valuable task for many medical applications. Marker-based optoelectronic systems are considered the gold standard in human motion tracking. However, their use is not always feasible in clinics and industrial environments. On the other hand, marker-less sensors became valuable tools, as they are inexpensive, noninvasive and easy to use. However, their accuracy can depend on many factors including sensor positioning, light conditions and body occlusions. In this study, following previous works on the feasibility of marker-less systems for human motion monitoring, we investigate the performance of the Microsoft Azure Kinect sensor in computing kinematic and dynamic measurements of static postures and dynamic movements. According to our knowledge, it is the first time that this sensor is compared with a Vicon marker-based system to assess the best camera positioning while observing the upper body part movements of people performing several tasks. Twenty-five healthy volunteers were monitored to evaluate the effects of the several testing conditions, including the Azure Kinect positions, the light conditions, and lower limbs occlusions, on the tracking accuracy of kinematic, dynamic, and motor control parameters. From the statistical analysis of the performed measurements, the camera in the frontal position was the most reliable, the lighting conditions had almost no effects on the tracking accuracy, while the lower limbs occlusion worsened the accuracy of the upper limbs. The assessment of human static postures and dynamic movements based on experimental data proves the feasibility of applying the Azure Kinect to the biomechanical monitoring of human motion in several fields.

Behavioral patterns in robotic collaborative assembly: comparing neurotypical and Autism Spectrum Disorder participants.

Introduction: In Industry 4.0, collaborative tasks often involve operators working with collaborative robots (cobots) in shared workspaces. Many aspects of the operator's well-being within this environment still need in-depth research. Moreover, these aspects are expected to differ between neurotypical (NT) and Autism Spectrum Disorder (ASD) operators. Methods: This study examines behavioral patterns in 16 participants (eight neurotypical, eight with high-functioning ASD) during an assembly task in an industry-like lab-based robotic collaborative cell, enabling the detection of potential risks to their well-being during industrial human-robot collaboration. Each participant worked on the task for five consecutive days, 3.5 h per day. During these sessions, six video clips of 10 min each were recorded for each participant. The videos were used to extract quantitative behavioral data using the NOVA annotation tool and analyzed qualitatively using an ad-hoc observational grid. Also, during the work sessions, the researchers took unstructured notes of the observed behaviors that were analyzed qualitatively. Results: The two groups differ mainly regarding behavior (e.g., prioritizing the robot partner, gaze patterns, facial expressions, multi-tasking, and personal space), adaptation to the task over time, and the resulting overall performance. Discussion: This result confirms that NT and ASD participants in a collaborative shared workspace have different needs and that the working experience should be tailored depending on the end-user's characteristics. The findings of this study represent a starting point for further efforts to promote well-being in the workplace. To the best of our knowledge, this is the first work comparing NT and ASD participants in a collaborative industrial scenario.

Biomechanical Assessments of the Upper Limb for Determining Fatigue, Strain and Effort from the Laboratory to the Industrial Working Place: A Systematic Review.

Recent human-centered developments in the industrial field (Industry 5.0) lead companies and stakeholders to ensure the wellbeing of their workers with assessments of upper limb performance in the workplace, with the aim of reducing work-related diseases and improving awareness of the physical status of workers, by assessing motor performance, fatigue, strain and effort. Such approaches are usually developed in laboratories and only at times they are translated to on-field applications; few studies summarized common practices for the assessments. Therefore, our aim is to review the current state-of-the-art approaches used for the assessment of fatigue, strain and effort in working scenarios and to analyze in detail the differences between studies that take place in the laboratory and in the workplace, in order to give insights on future trends and directions. A systematic review of the studies aimed at evaluating the motor performance, fatigue, strain and effort of the upper limb targeting working scenarios is presented. A total of 1375 articles were found in scientific databases and 288 were analyzed. About half of the scientific articles are focused on laboratory pilot studies investigating effort and fatigue in laboratories, while the other half are set in working places. Our results showed that assessing upper limb biomechanics is quite common in the field, but it is mostly performed with instrumental assessments in laboratory studies, while questionnaires and scales are preferred in working places. Future directions may be oriented towards multi-domain approaches able to exploit the potential of combined analyses, exploitation of instrumental approaches in workplace, targeting a wider range of people and implementing more structured trials to translate pilot studies to real practice.

The effects of robotic assistance on upper limb spatial muscle synergies in healthy people during planar upper-limb training.

BACKGROUND: Robotic rehabilitation is a commonly adopted technique used to restore motor functionality of neurological patients. However, despite promising results were achieved, the effects of human-robot interaction on human motor control and the recovery mechanisms induced with robot assistance can be further investigated even on healthy subjects before translating to clinical practice. In this study, we adopt a standard paradigm for upper-limb rehabilitation (a planar device with assistive control) with linear and challenging curvilinear trajectories to investigate the effect of the assistance in human-robot interaction in healthy people. METHODS: Ten healthy subjects were instructed to perform a large set of radial and curvilinear movements in two interaction modes: 1) free movement (subjects hold the robot handle with no assistance) and 2) assisted movement (with a force tunnel assistance paradigm). Kinematics and EMGs from representative upper-limb muscles were recorded to extract phasic muscle synergies. The free and assisted interaction modes were compared assessing the level of assistance, error, and muscle synergy comparison between the two interaction modes. RESULTS: It was found that in free movement error magnitude is higher than with assistance, proving that task complexity required assistance also on healthy controls. Moreover, curvilinear tasks require more assistance than standard radial paths and error is higher. Interestingly, while assistance improved task performance, we found only a slight modification of phasic synergies when comparing assisted and free movement. CONCLUSIONS: We found that on healthy people, the effect of assistance was significant on task performance, but limited on muscle synergies. The findings of this study can find applications for assessing human-robot interaction and to design training to maximize motor recovery.

Optimal Biomechanical Performance in Upper-Limb Gestures Depends on Velocity and Carried Load.

In the last few years, there has been increased interest in the preservation of physical and mental health of workers that cooperate with robots in industrial contexts, such as in the framework of the European H2020 Mindbot Project. Since biomechanical analysis contributes to the characterization of the subject interacting with a robotic setup and platform, we tested different speed and loading conditions in a simulated environment to determine upper-limb optimal performance. The simulations were performed starting from laboratory data of people executing upper-limb frontal reaching movements, by scaling the motion law and imposing various carried loads at the hand. The simulated velocity ranged from 20% to 200% of the original natural speed, with step increments of 10%, while the hand loads were 0, 0.5, 1, and 2 kg, simulating carried objects. A 3D inverse kinematic and dynamic model was used to compute upper-limb kinematics and dynamics, including shoulder flexion, shoulder abduction, and elbow flexion. An optimal range of velocities was found in which the expended energy was lower. Interestingly, the optimal speed corresponding to lower exerted torque and energy decreased when the load applied increased. Lastly, we introduced a preliminary movement inefficiency index to evaluate the deviation of the power and expended energy for the shoulder flexion degree of freedom when not coinciding with the minimum energy condition. These results can be useful in human-robot collaboration to design minimum-fatigue collaborative tasks, tune setup parameters and robot behavior, and support physical and mental health for workers.

Robotic Assistance for Upper Limbs May Induce Slight Changes in Motor Modules Compared With Free Movements in Stroke Survivors: A Cluster-Based Muscle Synergy Analysis.

Background: The efficacy of robot-assisted rehabilitation as a technique for achieving motor recovery is still being debated. The effects of robotic assistance are generally measured using standard clinical assessments. Few studies have investigated the value of human-centered instrumental analysis, taking the modular organization of the human neuromotor system into account in assessing how stroke survivors interact with robotic set-ups. In this paper, muscle synergy analysis was coupled with clustering procedures to elucidate the effect of human-robot interaction on the spatial and temporal features, and directional tuning of motor modules during robot-assisted movements. Methods: Twenty-two stroke survivors completed a session comprising a series of hand-to-mouth movements with and without robotic assistance. Patients were assessed instrumentally, recording kinematic, and electromyographic data to extract spatial muscle synergies and their temporal components. Patients' spatial synergies were grouped by means of a cluster analysis, matched pairwise across conditions (free and robot-assisted movement), and compared in terms of their spatial and temporal features, and directional tuning, to examine how robotic assistance altered their motor modules. Results: Motor synergies were successfully extracted for all 22 patients in both conditions. Seven clusters (spatial synergies) could describe the original datasets, in both free and robot-assisted movements. Interacting with the robot slightly altered the spatial synergies' features (to a variable extent), as well as their temporal components and directional tuning. Conclusions: Slight differences were identified in the characteristics of spatial synergies, temporal components and directional tuning of the motor modules of stroke survivors engaging in free and robot-assisted movements. Such effects are worth investigating in the framework of a modular description of the neuromusculoskeletal system to shed more light on human-robot interaction, and the effects of robotic assistance and rehabilitation.

Assessing User Transparency with Muscle Synergies during Exoskeleton-Assisted Movements: A Pilot Study on the LIGHTarm Device for Neurorehabilitation.

Exoskeleton devices for upper limb neurorehabilitation are one of the most exploited solutions for the recovery of lost motor functions. By providing weight support, passively compensated exoskeletons allow patients to experience upper limb training. Transparency is a desirable feature of exoskeletons that describes how the device alters free movements or interferes with spontaneous muscle patterns. A pilot study on healthy subjects was conducted to evaluate the feasibility of assessing transparency in the framework of muscle synergies. For such purpose, the LIGHTarm exoskeleton prototype was used. LIGHTarm provides gravity support to the upper limb during the execution of movements in the tridimensional workspace. Surface electromyography was acquired during the execution of three daily life movements (reaching, hand-to-mouth, and hand-to-nape) in three different conditions: free movement, exoskeleton-assisted (without gravity compensation), and exoskeleton-assisted (with gravity compensation) on healthy people. Preliminary results suggest that the muscle synergy framework may provide valuable assessment of user transparency and weight support features of devices aimed at rehabilitation.

Kinect V2 implementation and testing of the reaching performance scale for motor evaluation of patients with neurological impairment.

Automated procedures for neurological patients' motor evaluation may take advantage of the coupling between clinical scales and motion tracking devices to provide affordable, quantified and reliable assessment to be used in clinics, in surgeries and domestic environment. In this study, 20 post-stroke patients performed frontal reaching movements with their more affected limb, and a physician administered the Reaching Performance Scale (RPS) to assess motor functionality. At the same time, patients' kinematics were recorded with the Kinect V2 sensor. An automated algorithm was developed to compute the RPS based on Kinect V2 tracking data, and visual and Kinect V2 RPS scores were compared. Results showed very high statistical correlation between the automated procedure and the visual administration (Pearson Correlation Coefficient: R = 0.90, p < 0.001). While the number of patients is limited, the automated RPS seems to be successfully applicable to different levels of impairment, from mild to severe.

DUALarm: An open-source and 3D-printable device for upper limb neurorehabilitation.

Positively advocating that low-cost additive 3D-printing technologies and open-source licensed software/hardware platforms represent an optimal solution to realize low-cost equipment, a mechanical and 3D-printable device for bilateral upper-limb rehabilitation is presented. The design and manufacturing process of this wheel-geared mechanism, enabling in-phase and anti-phase movements, will be openly provided online with the aim of making a set of customizable devices for neurorehabilitation exploitable all over the world even by people/countries with limited economical and technological resources. In order to characterize the interaction with the device, preliminary trials with EMG and kinematics recordings were performed on healthy subjects.

Muscle Synergies-Based Characterization and Clustering of Poststroke Patients in Reaching Movements.

BACKGROUND: A deep characterization of neurological patients is a crucial step for a detailed knowledge of the pathology and maximal exploitation and customization of the rehabilitation therapy. The muscle synergies analysis was designed to investigate how muscles coactivate and how their eliciting commands change in time during movement production. Few studies investigated the value of muscle synergies for the characterization of neurological patients before rehabilitation therapies. In this article, the synergy analysis was used to characterize a group of chronic poststroke hemiplegic patients. METHODS: Twenty-two poststroke patients performed a session composed of a sequence of 3D reaching movements. They were assessed through an instrumental assessment, by recording kinematics and electromyography to extract muscle synergies and their activation commands. Patients' motor synergies were grouped by the means of cluster analysis. Consistency and characterization of each cluster was assessed and clinically profiled by comparison with standard motor assessments. RESULTS: Motor synergies were successfully extracted on all 22 patients. Five basic clusters were identified as a trade-off between clustering precision and synthesis power, representing: healthy-like activations, two shoulder compensatory strategies, two elbow predominance patterns. Each cluster was provided with a deep characterization and correlation with clinical scales, range of motion, and smoothness. CONCLUSION: The clustering of muscle synergies enabled a pretherapy characterization of patients. Such technique may affect several aspects of the therapy: prediction of outcomes, evaluation of the treatments, customization of doses, and therapies.

Kinect V2 Performance Assessment in Daily-Life Gestures: Cohort Study on Healthy Subjects for a Reference Database for Automated Instrumental Evaluations on Neurological Patients.

BACKGROUND: The increase of sanitary costs related to poststroke rehabilitation requires new sustainable and cost-effective strategies for promoting autonomous and dehospitalized motor training. In the Riprendo@Home and Future Home for Future Communities research projects, the promising approach of introducing low-cost technologies that promote home rehabilitation is exploited. In order to provide reliable evaluation of patients, a reference database of healthy people's performances is required and should consider variability related to healthy people performances. METHODS: 78 healthy subjects performed several repetitions of daily-life gestures, the reaching movement (RM) and hand-to-mouth (HtMM) movement with both the dominant and nondominant upper limbs. Movements were recorded with a Kinect V2. A synthetic biomechanical protocol based on kinematical, dynamical, and motor control parameters was used to assess motor performance of the healthy people. The investigation was conducted by clustering participants depending on their limb dominancy (right/left), gender (male/female), and age (young/middle/senior) as sources of variability. RESULTS: Results showed that limb dominancy has minor relevance in affecting RM and HtMM; gender has relevance in affecting the HtMM; age has major effect in affecting RM and HtMM. CONCLUSIONS: An investigation of healthy subjects' upper limb performances during daily-life gestures was performed with the Kinect V2 sensor. Findings will be the basis for a database of normative data for neurological patients' motor evaluation.

An affordable, adaptable, and hybrid assistive device for upper-limb neurorehabilitation.

The paper presents a multisensory and multimodal device for neuromuscular rehabilitation of the upper limb, designed to enable enriched rehabilitation treatment in both clinical and home environments. Originating from an existing low-cost, variable-stiffness rehabilitation device, it expands its functionalities by integrating additional modules in order to augment application scenarios and applicable clinical techniques. The newly developed system focuses on the integration of a wearable neuromuscular electrical stimulation system, a virtual rehabilitation scenario, a low-cost unobtrusive sensory system and a patient model for adapting training task parameters. It also monitors the user behavior during each single session and its evolution throughout the entire training period. The result is a modular, integrated and affordable rehabilitation device, enabling a biomechanical, neurological, and physiological-based training of patients, including innovative features currently unavailable within off-the-shelf rehabilitation devices.

Normative Data for an Instrumental Assessment of the Upper-Limb Functionality.

Upper-limb movement analysis is important to monitor objectively rehabilitation interventions, contributing to improving the overall treatments outcomes. Simple, fast, easy-to-use, and applicable methods are required to allow routinely functional evaluation of patients with different pathologies and clinical conditions. This paper describes the Reaching and Hand-to-Mouth Evaluation Method, a fast procedure to assess the upper-limb motor control and functional ability, providing a set of normative data from 42 healthy subjects of different ages, evaluated for both the dominant and the nondominant limb motor performance. Sixteen of them were reevaluated after two weeks to perform test-retest reliability analysis. Data were clustered into three subgroups of different ages to test the method sensitivity to motor control differences. Experimental data show notable test-retest reliability in all tasks. Data from older and younger subjects show significant differences in the measures related to the ability for coordination thus showing the high sensitivity of the method to motor control differences. The presented method, provided with control data from healthy subjects, appears to be a suitable and reliable tool for the upper-limb functional assessment in the clinical environment.

Kinect One-based biomechanical assessment of upper-limb performance compared to clinical scales in post-stroke patients.

This paper presents a Kinect One sensor-based protocol for the evaluation of the motor-performances of the upper limb of neurological patients during rehabilitative sessions. The assessment provides evaluations of kinematic, dynamic, motor and postural control variables. A pilot study was conducted on three post-stroke neurological patients, comparing Kinect-One biomechanical assessment with the outcomes of some of the most common clinical scales for the evaluation of the upper-limb functionality. Preliminary results indicate coherency between the clinical and instrumental evaluation. Moreover, the Kinect-One assessment seems to provide some complementary quantitative information, consistently integrating the clinical assessment.

SafeNet: a methodology for integrating general-purpose unsafe devices in safe-robot rehabilitation systems.

Robot-assisted neurorehabilitation often involves networked systems of sensors ("sensory rooms") and powerful devices in physical interaction with weak users. Safety is unquestionably a primary concern. Some lightweight robot platforms and devices designed on purpose include safety properties using redundant sensors or intrinsic safety design (e.g. compliance and backdrivability, limited exchange of energy). Nonetheless, the entire "sensory room" shall be required to be fail-safe and safely monitored as a system at large. Yet, sensor capabilities and control algorithms used in functional therapies require, in general, frequent updates or re-configurations, making a safety-grade release of such devices hardly sustainable in cost-effectiveness and development time. As such, promising integrated platforms for human-in-the-loop therapies could not find clinical application and manufacturing support because of lacking in the maintenance of global fail-safe properties. Under the general context of cross-machinery safety standards, the paper presents a methodology called SafeNet for helping in extending the safety rate of Human Robot Interaction (HRI) systems using unsafe components, including sensors and controllers. SafeNet considers, in fact, the robotic system as a device at large and applies the principles of functional safety (as in ISO 13489-1) through a set of architectural procedures and implementation rules. The enabled capability of monitoring a network of unsafe devices through redundant computational nodes, allows the usage of any custom sensors and algorithms, usually planned and assembled at therapy planning-time rather than at platform design-time. A case study is presented with an actual implementation of the proposed methodology. A specific architectural solution is applied to an example of robot-assisted upper-limb rehabilitation with online motion tracking.

The kinematic architecture of the Active Headframe: A new head support for awake brain surgery.

This paper presents the novel hybrid kinematic structure of the Active Headframe, a robotic head support to be employed in brain surgery operations for an active and dynamic control of the patient's head position and orientation, particularly addressing awake surgery requirements. The topology has been conceived in order to satisfy all the installation, functional and dynamic requirements. A kinetostatic optimization has been performed to obtain the actual geometric dimensions of the prototype currently being developed.

A spherical parallel three degrees-of-freedom robot for ankle-foot neuro-rehabilitation.

The ankle represents a fairly complex bone structure, resulting in kinematics that hinders a flawless robot-assisted recovery of foot motility in impaired subjects. The paper proposes a novel device for ankle-foot neuro-rehabilitation based on a mechatronic redesign of the remarkable Agile Eye spherical robot on the basis of clinical requisites. The kinematic design allows the positioning of the ankle articular center close to the machine rotation center with valuable benefits in term of therapy functions. The prototype, named PKAnkle, Parallel Kinematic machine for Ankle rehabilitation, provides a 6-axes load cell for the measure of subject interaction forces/torques, and it integrates a commercial EMG-acquisition system. Robot control provides active and passive therapeutic exercises.

Analysis of elbow-joints misalignment in upper-limb exoskeleton.

This paper presents advantages of introducing elbow-joints misalignments in an exoskeleton for upper limb rehabilitation. Typical exoskeletons are characterized by axes of the device as much as possible aligned to the rotational axes of human articulations. This approach leads to advantages in terms of movements and torques decoupling, but can lead to limitations nearby the elbow singular configuration. A proper elbow axes misalignment between the exoskeleton and the human can improve the quality of collaborative rehabilitation therapies, in which a correct torque transmission from human articulations to mechanical joints of the device is required to react to torques generated by the patient.