Hand Gesture Interface for Robot Path Definition in Collaborative Applications: Implementation and Comparative Study.

The article explores the possibilities of using hand gestures as a control interface for robotic systems in a collaborative workspace. The development of hand gesture control interfaces has become increasingly important in everyday life as well as professional contexts such as manufacturing processes. We present a system designed to facilitate collaboration between humans and robots in manufacturing processes that require frequent revisions of the robot path and that allows direct definition of the waypoints, which differentiates our system from the existing ones. We introduce a novel and intuitive approach to human-robot cooperation through the use of simple gestures. As part of a robotic workspace, a proposed interface was developed and implemented utilising three RGB-D sensors for monitoring the operator's hand movements within the workspace. The system employs distributed data processing through multiple Jetson Nano units, with each unit processing data from a single camera. MediaPipe solution is utilised to localise the hand landmarks in the RGB image, enabling gesture recognition. We compare the conventional methods of defining robot trajectories with their developed gesture-based system through an experiment with 20 volunteers. The experiment involved verification of the system under realistic conditions in a real workspace closely resembling the intended industrial application. Data collected during the experiment included both objective and subjective parameters. The results indicate that the gesture-based interface enables users to define a given path objectively faster than conventional methods. We critically analyse the features and limitations of the developed system and suggest directions for future research. Overall, the experimental results indicate the usefulness of the developed system as it can speed up the definition of the robot's path.

Distributed Camera Subsystem for Obstacle Detection.

This work focuses on improving a camera system for sensing a workspace in which dynamic obstacles need to be detected. The currently available state-of-the-art solution (MoveIt!) processes data in a centralized manner from cameras that have to be registered before the system starts. Our solution enables distributed data processing and dynamic change in the number of sensors at runtime. The distributed camera data processing is implemented using a dedicated control unit on which the filtering is performed by comparing the real and expected depth images. Measurements of the processing speed of all sensor data into a global voxel map were compared between the centralized system (MoveIt!) and the new distributed system as part of a performance benchmark. The distributed system is more flexible in terms of sensitivity to a number of cameras, better framerate stability and the possibility of changing the camera number on the go. The effects of voxel grid size and camera resolution were also compared during the benchmark, where the distributed system showed better results. Finally, the overhead of data transmission in the network was discussed where the distributed system is considerably more efficient. The decentralized system proves to be faster by 38.7% with one camera and 71.5% with four cameras.

Finding the Optimal Pose of 2D LLT Sensors to Improve Object Pose Estimation.

In this paper, we examine a method for improving pose estimation by correctly positioning the sensors relative to the scanned object. Three objects made of different materials and using different manufacturing technologies were selected for the experiment. To collect input data for orientation estimation, a simulation environment was created where each object was scanned at different poses. A simulation model of the laser line triangulation sensor was created for scanning, and the optical surface properties of the scanned objects were set to simulate real scanning conditions. The simulation was verified on a real system using the UR10e robot to rotate and move the object. The presented results show that the simulation matches the real measurements and that the appropriate placement of the sensors has improved the orientation estimation.

Camera Arrangement Optimization for Workspace Monitoring in Human-Robot Collaboration.

Human-robot interaction is becoming an integral part of practice. There is a greater emphasis on safety in workplaces where a robot may bump into a worker. In practice, there are solutions that control the robot based on the potential energy in a collision or a robot re-planning the straight-line trajectory. However, a sensor system must be designed to detect obstacles across the human-robot shared workspace. So far, there is no procedure that engineers can follow in practice to deploy sensors ideally. We come up with the idea of classifying the space as an importance index, which determines what part of the workspace sensors should sense to ensure ideal obstacle sensing. Then, the ideal camera positions can be automatically found according to this classified map. Based on the experiment, the coverage of the important volume by the calculated camera position in the workspace was found to be on average 37% greater compared to a camera placed intuitively by test subjects. Using two cameras at the workplace, the calculated positions were 27% more effective than the subjects' camera positions. Furthermore, for three cameras, the calculated positions were 13% better than the subjects' camera positions, with a total coverage of more than 99% of the classified map.

Using Virtual Scanning to Find Optimal Configuration of a 3D Scanner Turntable for Scanning of Mechanical Parts.

The article describes a method of simulated 3D scanning of triangle meshes based on ray casting which is used to find the optimal configuration of a real 3D scanner turntable. The configuration include the number of scanners, their elevation above the rotary table and the number of required rotation steps. The evaluation is based on the percentage of the part surface covered by the resulting point cloud, which determines the ability to capture all details of the shape. Principal component analysis is used as a secondary criterion to also evaluate the ability to capture the overall general proportions of the model.

Increasing the Reliability of Data Collection of Laser Line Triangulation Sensor by Proper Placement of the Sensor.

In this paper, we investigated the effect of the incidence angle of a laser ray on the reflected laser intensity. A dataset on this dependence is presented for materials usually used in the industry, such as transparent and non-transparent plastics and aluminum alloys with different surface roughness. The measurements have been performed with a laser line triangulation sensor and a UR10e robot. The presented results are proposing where to place the sensor relative to the scanned object, thus increasing the reliability of the sensor data collection.

Analysis of Precision and Stability of Hand Tracking with Leap Motion Sensor.

In this analysis, we present results from measurements performed to determine the stability of a hand tracking system and the accuracy of the detected palm and finger's position. Measurements were performed for the evaluation of the sensor for an application in an industrial robot-assisted assembly scenario. Human-robot interaction is a relevant topic in collaborative robotics. Intuitive and straightforward control tools for robot navigation and program flow control are essential for effective utilisation in production scenarios without unnecessary slowdowns caused by the operator. For the hand tracking and gesture-based control, it is necessary to know the sensor's accuracy. For gesture recognition with a moving target, the sensor must provide stable tracking results. This paper evaluates the sensor's real-world performance by measuring the localisation deviations of the hand being tracked as it moves in the workspace.

Verification of electronic device technology for measurement and evaluation of thermal exposure of fire fighters and members of rescue teams.

BACKGROUND: The work of members of rescue teams could be associated with very high physical and thermal loads. If not timely interrupted, any extreme labour-thermal load may lead to a failure of the body and fatal collapse. This risk may be significantly reduced by devices that monitor the response of the body during the intervention and inform rescuers about the need to interrupt the exposure when the critical value of the reference indicator is achieved. The aim of the study was to test the correlation between the data of the newly developed device for signaling the strain of rescuers and the indicators of physiological response of the body. MATERIAL AND METHODS: The tests were performed on 2 physically fit fire fighters dressed in a protective rescue suit and using insulating breathing apparatus, over a wide range of heat load under a model load on a bicycle ergometer in a climatic chamber. RESULTS: The study provided a significant correlation between the body temperature measured in the ear canal and the temperature under the suit sensed by the tested device - the Safety Ambient Monitor (SAM) (R = 0.9007). The temperature under the suit also correlated with the temperature of the chest skin (R = 0.8928) and heart rate (R = 0.8613). CONCLUSIONS: A statistically significant correlation was proven between the temperature sensed by the SAM and the body temperature. The technical solution of sensing the temperature under the suit using the verified SAM technology does not affect or limit fire fighters in their work and minimizes the possibility of damage to the sensor and signaling failures. Med Pr 2018;69(1):1-11.