SPT: Single Pedestrian Tracking Framework with Re-Identification-Based Learning Using the Siamese Model.

Pedestrian tracking is a challenging task in the area of visual object tracking research and it is a vital component of various vision-based applications such as surveillance systems, human-following robots, and autonomous vehicles. In this paper, we proposed a single pedestrian tracking (SPT) framework for identifying each instance of a person across all video frames through a tracking-by-detection paradigm that combines deep learning and metric learning-based approaches. The SPT framework comprises three main modules: detection, re-identification, and tracking. Our contribution is a significant improvement in the results by designing two compact metric learning-based models using Siamese architecture in the pedestrian re-identification module and combining one of the most robust re-identification models for data associated with the pedestrian detector in the tracking module. We carried out several analyses to evaluate the performance of our SPT framework for single pedestrian tracking in the videos. The results of the re-identification module validate that our two proposed re-identification models surpass existing state-of-the-art models with increased accuracies of 79.2% and 83.9% on the large dataset and 92% and 96% on the small dataset. Moreover, the proposed SPT tracker, along with six state-of-the-art (SOTA) tracking models, has been tested on various indoor and outdoor video sequences. A qualitative analysis considering six major environmental factors verifies the effectiveness of our SPT tracker under illumination changes, appearance variations due to pose changes, changes in target position, and partial occlusions. In addition, quantitative analysis based on experimental results also demonstrates that our proposed SPT tracker outperforms the GOTURN, CSRT, KCF, and SiamFC trackers with a success rate of 79.7% while beating the DiamSiamRPN, SiamFC, CSRT, GOTURN, and SiamMask trackers with an average of 18 tracking frames per second.

Development of an Autonomous Driving Vehicle for Garbage Collection in Residential Areas.

Autonomous driving and its real-world implementation have been among the most actively studied topics in the past few years. In recent years, this growth has been accelerated by the development of advanced deep learning-based data processing technologies. Moreover, large automakers manufacture vehicles that can achieve partially or fully autonomous driving for driving on real roads. However, self-driving cars are limited to some areas with multi-lane roads, such as highways, and self-driving cars that drive in urban areas or residential complexes are still in the development stage. Among autonomous vehicles for various purposes, this paper focused on the development of autonomous vehicles for garbage collection in residential areas. Since we set the target environment of the vehicle as a residential complex, there is a difference from the target environment of a general autonomous vehicle. Therefore, in this paper, we defined ODD, including vehicle length, speed, and driving conditions for the development vehicle to drive in a residential area. In addition, to recognize the vehicle's surroundings and respond to various situations, it is equipped with various sensors and additional devices that can notify the outside of the vehicle's state or operate it in an emergency. In addition, an autonomous driving system capable of object recognition, lane recognition, route planning, vehicle manipulation, and abnormal situation detection was configured to suit the vehicle hardware and driving environment configured in this way. Finally, by performing autonomous driving in the actual experimental section with the developed vehicle, it was confirmed that the function of autonomous driving in the residential area works appropriately. Moreover, we confirmed that this vehicle would support garbage collection works through the experiment of work efficiency.

3D Recognition Based on Sensor Modalities for Robotic Systems: A Survey.

3D visual recognition is a prerequisite for most autonomous robotic systems operating in the real world. It empowers robots to perform a variety of tasks, such as tracking, understanding the environment, and human-robot interaction. Autonomous robots equipped with 3D recognition capability can better perform their social roles through supportive task assistance in professional jobs and effective domestic services. For active assistance, social robots must recognize their surroundings, including objects and places to perform the task more efficiently. This article first highlights the value-centric role of social robots in society by presenting recently developed robots and describes their main features. Instigated by the recognition capability of social robots, we present the analysis of data representation methods based on sensor modalities for 3D object and place recognition using deep learning models. In this direction, we delineate the research gaps that need to be addressed, summarize 3D recognition datasets, and present performance comparisons. Finally, a discussion of future research directions concludes the article. This survey is intended to show how recent developments in 3D visual recognition based on sensor modalities using deep-learning-based approaches can lay the groundwork to inspire further research and serves as a guide to those who are interested in vision-based robotics applications.

A Classification-Lock Tracking Strategy Allowing a Person-Following Robot to Operate in a Complicated Indoor Environment.

Person-following technology is an important robot service. The major trend of person-following is to utilize computer vision technology to localize the target person, due to the wide view and rich information that is obtained from the real world through a camera. However, most existing approaches employ the detecting-by-tracking strategy, which suffers from low speed, accompanied with more complicated detecting models and unstable region of interest (ROI) outputs in unexpressed situations. In this paper, we propose a novel classification-lock strategy to localize the target person, which incorporates the visual tracking technology with object detection technology, to adapt the localization model to different environments online, and to keep a high frame-per-second (FPS) on the mobile platform. This person-following approach consists of three key parts. In the first step, a pairwise cluster tracker is employed to localize the person. A positive and negative classifier is then utilized to verify the tracker's result and to update the tracking model. In addition, a detector pre-trained by a CPU-optimized convolutional neural network is used to further improve the result of tracking. In the experiment, our approach is compared with other state-of-art approaches by a Vojir tracking dataset, with three sequences in the items of human to prove the quality of person localization. Moreover, the common challenges during the following task are evaluated by several image sequences in a static scene, and a dynamic scene is used to evaluate the improvement from the classification-lock strategy. Finally, our approach is deployed on a mobile robot to test its performance on the function of the person-following. Compared with other state-of-art methods, our approach achieves the highest score (0.91 recall rate). In the static and dynamic scene, the output of the ROI based on the classification-lock strategy is significantly better than that without it. Our approach also succeeds in a long-term following task in an indoor multi-floor scenario.

Fusing range measurements from ultrasonic beacons and a laser range finder for localization of a mobile robot.

This paper proposes a method for mobile robot localization in a partially unknown indoor environment. The method fuses two types of range measurements: the range from the robot to the beacons measured by ultrasonic sensors and the range from the robot to the walls surrounding the robot measured by a laser range finder (LRF). For the fusion, the unscented Kalman filter (UKF) is utilized. Because finding the Jacobian matrix is not feasible for range measurement using an LRF, UKF has an advantage in this situation over the extended KF. The locations of the beacons and range data from the beacons are available, whereas the correspondence of the range data to the beacon is not given. Therefore, the proposed method also deals with the problem of data association to determine which beacon corresponds to the given range data. The proposed approach is evaluated using different sets of design parameter values and is compared with the method that uses only an LRF or ultrasonic beacons. Comparative analysis shows that even though ultrasonic beacons are sparsely populated, have a large error and have a slow update rate, they improve the localization performance when fused with the LRF measurement. In addition, proper adjustment of the UKF design parameters is crucial for full utilization of the UKF approach for sensor fusion. This study contributes to the derivation of a UKF-based design methodology to fuse two exteroceptive measurements that are complementary to each other in localization.

Position and Velocity Estimation for Two-Inertia System with Nonlinear Stiffness Based on Acceleration Sensor.

In this paper, we consider the state estimation problem for flexible joint manipulators that involve nonlinear characteristics in their stiffness. The two key ideas of our design are that (a) an accelerometer is used in order that the estimation error dynamics do not depend on nonlinearities at the link part of the manipulators and (b) the model of the nonlinear stiffness is indeed a Lipschitz function. Based on the measured acceleration, we propose a nonlinear observer under the Lipschitz condition of the nonlinear stiffness. In addition, in order to effectively compensate for the estimation error, the gain of the proposed observer is chosen from the ARE (algebraic Riccati equations) which depend on the Lipschitz constant. Comparative experimental results verify the effectiveness of the proposed method.