Cyclist Orientation Estimation Using LiDAR Data.

It is crucial for an autonomous vehicle to predict cyclist behavior before decision-making. When a cyclist is on real traffic roads, his or her body orientation indicates the current moving directions, and his or her head orientation indicates his or her intention for checking the road situation before making next movement. Therefore, estimating the orientation of cyclist's body and head is an important factor of cyclist behavior prediction for autonomous driving. This research proposes to estimate cyclist orientation including both body and head orientation using deep neural network with the data from Light Detection and Ranging (LiDAR) sensor. In this research, two different methods are proposed for cyclist orientation estimation. The first method uses 2D images to represent the reflectivity, ambient and range information collected by LiDAR sensor. At the same time, the second method uses 3D point cloud data to represent the information collected from LiDAR sensor. The two proposed methods adopt a model ResNet50, which is a 50-layer convolutional neural network, for orientation classification. Hence, the performances of two methods are compared to achieve the most effective usage of LiDAR sensor data in cyclist orientation estimation. This research developed a cyclist dataset, which includes multiple cyclists with different body and head orientations. The experimental results showed that a model that uses 3D point cloud data has better performance for cyclist orientation estimation compared to the model that uses 2D images. Moreover, in the 3D point cloud data-based method, using reflectivity information has a more accurate estimation result than using ambient information.

Pedestrian Detection Using Multispectral Images and a Deep Neural Network.

Pedestrian fatalities and injuries most likely occur in vehicle-pedestrian crashes. Meanwhile, engineers have tried to reduce the problems by developing a pedestrian detection function in Advanced Driver-Assistance Systems (ADAS) and autonomous vehicles. However, the system is still not perfect. A remaining problem in pedestrian detection is the performance reduction at nighttime, although pedestrian detection should work well regardless of lighting conditions. This study presents an evaluation of pedestrian detection performance in different lighting conditions, then proposes to adopt multispectral image and deep neural network to improve the detection accuracy. In the evaluation, different image sources including RGB, thermal, and multispectral format are compared for the performance of the pedestrian detection. In addition, the optimizations of the architecture of the deep neural network are performed to achieve high accuracy and short processing time in the pedestrian detection task. The result implies that using multispectral images is the best solution for pedestrian detection at different lighting conditions. The proposed deep neural network accomplishes a 6.9% improvement in pedestrian detection accuracy compared to the baseline method. Moreover, the optimization for processing time indicates that it is possible to reduce 22.76% processing time by only sacrificing 2% detection accuracy.

Multi-Person Pose Estimation Using an Orientation and Occlusion Aware Deep Learning Network.

Image based human behavior and activity understanding has been a hot topic in the field of computer vision and multimedia. As an important part, skeleton estimation, which is also called pose estimation, has attracted lots of interests. For pose estimation, most of the deep learning approaches mainly focus on the joint feature. However, the joint feature is not sufficient, especially when the image includes multi-person and the pose is occluded or not fully visible. This paper proposes a novel multi-task framework for the multi-person pose estimation. The proposed framework is developed based on Mask Region-based Convolutional Neural Networks (R-CNN) and extended to integrate the joint feature, body boundary, body orientation and occlusion condition together. In order to further improve the performance of the multi-person pose estimation, this paper proposes to organize the different information in serial multi-task models instead of the widely used parallel multi-task network. The proposed models are trained on the public dataset Common Objects in Context (COCO), which is further augmented by ground truths of body orientation and mutual-occlusion mask. Experiments demonstrate the performance of the proposed method for multi-person pose estimation and body orientation estimation. The proposed method can detect 84.6% of the Percentage of Correct Keypoints (PCK) and has an 83.7% Correct Detection Rate (CDR). Comparisons further illustrate the proposed model can reduce the over-detection compared with other methods.

Passive Sensor Integration for Vehicle Self-Localization in Urban Traffic Environment.

This research proposes an accurate vehicular positioning system which can achieve lane-level performance in urban canyons. Multiple passive sensors, which include Global Navigation Satellite System (GNSS) receivers, onboard cameras and inertial sensors, are integrated in the proposed system. As the main source for the localization, the GNSS technique suffers from Non-Line-Of-Sight (NLOS) propagation and multipath effects in urban canyons. This paper proposes to employ a novel GNSS positioning technique in the integration. The employed GNSS technique reduces the multipath and NLOS effects by using the 3D building map. In addition, the inertial sensor can describe the vehicle motion, but has a drift problem as time increases. This paper develops vision-based lane detection, which is firstly used for controlling the drift of the inertial sensor. Moreover, the lane keeping and changing behaviors are extracted from the lane detection function, and further reduce the lateral positioning error in the proposed localization system. We evaluate the integrated localization system in the challenging city urban scenario. The experiments demonstrate the proposed method has sub-meter accuracy with respect to mean positioning error.

NLOS Correction/Exclusion for GNSS Measurement Using RAIM and City Building Models.

Currently, global navigation satellite system (GNSS) receivers can provide accurate and reliable positioning service in open-field areas. However, their performance in the downtown areas of cities is still affected by the multipath and none-line-of-sight (NLOS) receptions. This paper proposes a new positioning method using 3D building models and the receiver autonomous integrity monitoring (RAIM) satellite selection method to achieve satisfactory positioning performance in urban area. The 3D building model uses a ray-tracing technique to simulate the line-of-sight (LOS) and NLOS signal travel distance, which is well-known as pseudorange, between the satellite and receiver. The proposed RAIM fault detection and exclusion (FDE) is able to compare the similarity between the raw pseudorange measurement and the simulated pseudorange. The measurement of the satellite will be excluded if the simulated and raw pseudoranges are inconsistent. Because of the assumption of the single reflection in the ray-tracing technique, an inconsistent case indicates it is a double or multiple reflected NLOS signal. According to the experimental results, the RAIM satellite selection technique can reduce by about 8.4% and 36.2% the positioning solutions with large errors (solutions estimated on the wrong side of the road) for the 3D building model method in the middle and deep urban canyon environment, respectively.

Classifying real and imaginary finger press tasks on a P300-based brain-computer interface.

Brain computer interfaces based on P300 and sensory-motor rhythms are widely studied and recent advances show some interest in the combination of the two. In this paper, typical P300 paradigm is modified by adding animation guide of the finger press as a stimulus and by using different response strategies (silent counting and actual/imaginary left or right index finger press following the animation). Both P300 potentials and sensory-motor rhythms are directly exploited and discussed. Classification results showed that even under very demanding conditions, which was, 200 ms inter-stimulus interval of the P300 stimuli and actual/imaginary finger press once per 1.6s, the paradigm can evoke both P300 potentials and sensory-motor rhythms simultaneously. Actual finger press increased single trial P300 selection accuracy of different subjects by 5-29.5% compared with silent counting; imaginary finger press did not increase the P300 selection accuracy apparently for most subjects except the two who were very poor at counting task. This showed by using different interface design and adopting certain mental response strategies, the 'BCI illiteracy' may be cured. Also imaginary task had good performance of left versus right classification (with the best subject reached 81.1% of accuracy), which is an additional information that can be used to improve system performance.