Application of Somatosensory Camera in Robot Remote Experimental Teaching

ABSTRACT

This article proposes a method to remotely control the robot in the laboratory through the Kinect camera to solve the impact of the Covid-19 epidemic on the laboratory teaching experience which allows users to remotely control robots through their own body movements to understand the principles of robots. It is used to solve the problem of fewer students willing to participate in robot remote education. In this study, the Azure Kinect DK camera was used to collect the motion posture of the upper limbs of the human body. The Kinect camera calculates the frames of human arm joints' motion. The control system calculates the direction of motion of each joint of the human body based on the quaternion by mapping the heterogeneous human joints with the robot joints. Make the posture of the human arm swing correspond to the posture of the robot's movement. Thus, the robot in the laboratory can be driven remotely through Azure Kinect DK. By using the method described in this article, students use the camera's motion capture system to remotely manipulate the robot to grab some simple objects. Through the method described in this research, students can carry out some simple operations on the robots in the laboratory from remote. So it is convenient for students to understand the basic principles of robots and achieve the purpose of better remote experimental teaching. At the same time, students can get practical application of motor servo control, ergonomics, physical simulation engine, digital twin system, etc. © 2022 IEEE.