RESUMO
Legged robots are better able to adapt to different terrains compared with wheeled robots. However, traditional motion controllers suffer from extremely complex dynamics properties. Reinforcement learning (RL) helps to overcome the complications of dynamics design and calculation. In addition, the high autonomy of the RL controller results in a more robust response to complex environments and terrains compared with traditional controllers. However, RL algorithms are limited by the problems of convergence and training efficiency due to the complexity of the task. Learn and outperform the reference motion (LORM), an RL based framework for gait controlling of biped robot is proposed leveraging the prior knowledge of reference motion. The proposed trained agent outperformed the reference motion and existing motion-based methods. The RL environment was finely crafted for optimal performance, including the pruning of state space and action space, reward shaping, and design of episode criterion. Several improvements were implemented to further improve the training efficiency and performance including: random state initialization (RSI), the noise of joint angles, and a novel improvement based on symmetrization of gait. To validate the proposed method, the Darwin-op robot was set as the target platform and two different tasks were designed: (I) Walking as fast as possible and (II) Tracking specific velocity. In task (I), the proposed method resulted in the walking velocity of 0.488 m/s, with a 5.8 times improvement compared with the original traditional reference controller. The directional accuracy improved by 87.3%. The velocity performance achieved 2× compared with the rated max velocity and more than 8× compared with other recent works. To our knowledge, our work achieved the best velocity performance on the platform Darwin-op. In task (II), the proposed method achieved a tracking accuracy of over 95%. Different environments are introduced including plains, slopes, uneven terrains, and walking with external force, where the robot was expected to maintain walking stability with ideal speed and little direction deviation, to validate the performance and robustness of the proposed method.
RESUMO
As the average age of the population goes higher, the people undergo hip arthroplasty surgery to replace their stiff and painful damaged hip joints. To reduce the risk factor after the surgery we developed a visual aided system in previous works. In order to solve the integration problems of commercial camera with other sensors into the femoral head and to minimize the area and power consumption, in this paper we propose a CMOS image sensor of resolution 200 × 200 specially designed for the application in which each individual pixel measures around 15 µm × 15 µm in size and the image sensor chip size measures about 3.5 mm × 3.5 mm. The proposed sensor is simulated with the input current variations from 2 pA to 100 pA for the individual pixels and the corresponding measurements for each pixel range from 2 mV to 855 mV. Besides, we put forward a new method of pattern detection and recognition in the blood-covered situation, which provides an accurate segmentation of patterns from the blood. All the detected patterns are recognized by generating its right 9-bit binary ID required for the pose estimation calculation. Furthermore, to reduce system power consumption, we implement algorithms on FPGA to process the image data pixel by pixel and transmit it directly to the computer for post-processing. Experimental results show that the pattern detection rate goes as high as 99%, which is 5% better in accuracy compared to the top hat algorithm. The power consumption of the system is 213 mW, which is a 70% decrease compared to our previous work.
