Semantic and Geometric-Aware Day-to-Night Image Translation Network.

Autonomous driving systems heavily depend on perception tasks for optimal performance. However, the prevailing datasets are primarily focused on scenarios with clear visibility (i.e., sunny and daytime). This concentration poses challenges in training deep-learning-based perception models for environments with adverse conditions (e.g., rainy and nighttime). In this paper, we propose an unsupervised network designed for the translation of images from day-to-night to solve the ill-posed problem of learning the mapping between domains with unpaired data. The proposed method involves extracting both semantic and geometric information from input images in the form of attention maps. We assume that the multi-task network can extract semantic and geometric information during the estimation of semantic segmentation and depth maps, respectively. The image-to-image translation network integrates the two distinct types of extracted information, employing them as spatial attention maps. We compare our method with related works both qualitatively and quantitatively. The proposed method shows both qualitative and qualitative improvements in visual presentation over related work.

LiDAR-to-Radar Translation Based on Voxel Feature Extraction Module for Radar Data Augmentation.

In autonomous vehicles, the LiDAR and radar sensors are indispensable components for measuring distances to objects. While deep-learning-based algorithms for LiDAR sensors have been extensively proposed, the same cannot be said for radar sensors. LiDAR and radar share the commonality of measuring distances, but they are used in different environments. LiDAR tends to produce less noisy data and provides precise distance measurements, but it is highly affected by environmental factors like rain and fog. In contrast, radar is less impacted by environmental conditions but tends to generate noisier data. To reduce noise in radar data and enhance radar data augmentation, we propose a LiDAR-to-Radar translation method with a voxel feature extraction module, leveraging the fact that both sensors acquire data in a point-based manner. Because of the translation of high-quality LiDAR data into radar data, this becomes achievable. We demonstrate the superiority of our proposed method by acquiring and using data from both LiDAR and radar sensors in the same environment for validation.

Deep Voxelized Feature Maps for Self-Localization in Autonomous Driving.

Lane-level self-localization is essential for autonomous driving. Point cloud maps are typically used for self-localization but are known to be redundant. Deep features produced by neural networks can be used as a map, but their simple utilization could lead to corruption in large environments. This paper proposes a practical map format using deep features. We propose voxelized deep feature maps for self-localization, consisting of deep features defined in small regions. The self-localization algorithm proposed in this paper considers per-voxel residual and reassignment of scan points in each optimization iteration, which could result in accurate results. Our experiments compared point cloud maps, feature maps, and the proposed map from the self-localization accuracy and efficiency perspective. As a result, more accurate and lane-level self-localization was achieved with the proposed voxelized deep feature map, even with a smaller storage requirement compared with the other map formats.

Transportation Mode Detection Combining CNN and Vision Transformer with Sensors Recalibration Using Smartphone Built-In Sensors.

Transportation Mode Detection (TMD) is an important task for the Intelligent Transportation System (ITS) and Lifelog. TMD, using smartphone built-in sensors, can be a low-cost and effective solution. In recent years, many studies have focused on TMD, yet they support a limited number of modes and do not consider similar transportation modes and holding places, limiting further applications. In this paper, we propose a new network framework to realize TMD, which combines structural and spatial interaction features, and considers the weights of multiple sensors' contributions, enabling the recognition of eight transportation modes with four similar transportation modes and four holding places. First, raw data is segmented and transformed into a spectrum image and then ResNet and Vision Transformers (Vit) are used to extract structural and spatial interaction features, respectively. To consider the contribution of different sensors, the weights of each sensor are recalibrated using an ECA module. Finally, Multi-Layer Perceptron (MLP) is introduced to fuse these two different kinds of features. The performance of the proposed method is evaluated on the public Sussex-Huawei Locomotion-Transportation (SHL) dataset, and is found to outperform the baselines by at least 10%.

GAN-Based LiDAR Translation between Sunny and Adverse Weather for Autonomous Driving and Driving Simulation.

Autonomous driving requires robust and highly accurate perception technologies. Various deep learning algorithms based on only image processing satisfy this requirement, but few such algorithms are based on LiDAR. However, images are only one part of the perceptible sensors in an autonomous driving vehicle; LiDAR is also essential for the recognition of driving environments. The main reason why there exist few deep learning algorithms based on LiDAR is a lack of data. Recent translation technology using generative adversarial networks (GANs) has been proposed to deal with this problem. However, these technologies focus on only image-to-image translation, although a lack of data occurs more often with LiDAR than with images. LiDAR translation technology is required not only for data augmentation, but also for driving simulation, which allows algorithms to practice driving as if they were commanding a real vehicle, before doing so in the real world. In other words, driving simulation is a key technology for evaluating and verifying algorithms which are practically applied to vehicles. In this paper, we propose a GAN-based LiDAR translation algorithm for autonomous driving and driving simulation. It is the first LiDAR translation approach that can deal with various types of weather that are based on an empirical approach. We tested the proposed method on the JARI data set, which was collected under various adverse weather scenarios with diverse precipitation and visible distance settings. The proposed method was also applied to the real-world Spain data set. Our experimental results demonstrate that the proposed method can generate realistic LiDAR data under adverse weather conditions.

Analysis of Occlusion Effects for Map-Based Self-Localization in Urban Areas.

A high-definition (HD) map provides structural information for map-based self-localization, enabling stable estimation in real environments. In urban areas, there are many obstacles, such as buses, that occlude sensor observations, resulting in self-localization errors. However, most of the existing HD map-based self-localization evaluations do not consider sudden significant errors due to obstacles. Instead, they evaluate this in terms of average error over estimated trajectories in an environment with few occlusions. This study evaluated the effects of self-localization estimation on occlusion with synthetically generated obstacles in a real environment. Various patterns of synthetic occlusion enabled the analyses of the effects of self-localization error from various angles. Our experiments showed various characteristics that locations susceptible to obstacles have. For example, we found that occlusion in intersections tends to increase self-localization errors. In addition, we analyzed the geometrical structures of a surrounding environment in high-level error cases and low-level error cases with occlusions. As a result, we suggested the concept that the real environment should have to achieve robust self-localization under occlusion conditions.

Pedestrian Detection Using Multispectral Images and a Deep Neural Network.

Pedestrian fatalities and injuries most likely occur in vehicle-pedestrian crashes. Meanwhile, engineers have tried to reduce the problems by developing a pedestrian detection function in Advanced Driver-Assistance Systems (ADAS) and autonomous vehicles. However, the system is still not perfect. A remaining problem in pedestrian detection is the performance reduction at nighttime, although pedestrian detection should work well regardless of lighting conditions. This study presents an evaluation of pedestrian detection performance in different lighting conditions, then proposes to adopt multispectral image and deep neural network to improve the detection accuracy. In the evaluation, different image sources including RGB, thermal, and multispectral format are compared for the performance of the pedestrian detection. In addition, the optimizations of the architecture of the deep neural network are performed to achieve high accuracy and short processing time in the pedestrian detection task. The result implies that using multispectral images is the best solution for pedestrian detection at different lighting conditions. The proposed deep neural network accomplishes a 6.9% improvement in pedestrian detection accuracy compared to the baseline method. Moreover, the optimization for processing time indicates that it is possible to reduce 22.76% processing time by only sacrificing 2% detection accuracy.

Multi-Person Pose Estimation Using an Orientation and Occlusion Aware Deep Learning Network.

Image based human behavior and activity understanding has been a hot topic in the field of computer vision and multimedia. As an important part, skeleton estimation, which is also called pose estimation, has attracted lots of interests. For pose estimation, most of the deep learning approaches mainly focus on the joint feature. However, the joint feature is not sufficient, especially when the image includes multi-person and the pose is occluded or not fully visible. This paper proposes a novel multi-task framework for the multi-person pose estimation. The proposed framework is developed based on Mask Region-based Convolutional Neural Networks (R-CNN) and extended to integrate the joint feature, body boundary, body orientation and occlusion condition together. In order to further improve the performance of the multi-person pose estimation, this paper proposes to organize the different information in serial multi-task models instead of the widely used parallel multi-task network. The proposed models are trained on the public dataset Common Objects in Context (COCO), which is further augmented by ground truths of body orientation and mutual-occlusion mask. Experiments demonstrate the performance of the proposed method for multi-person pose estimation and body orientation estimation. The proposed method can detect 84.6% of the Percentage of Correct Keypoints (PCK) and has an 83.7% Correct Detection Rate (CDR). Comparisons further illustrate the proposed model can reduce the over-detection compared with other methods.

Passive Sensor Integration for Vehicle Self-Localization in Urban Traffic Environment.

This research proposes an accurate vehicular positioning system which can achieve lane-level performance in urban canyons. Multiple passive sensors, which include Global Navigation Satellite System (GNSS) receivers, onboard cameras and inertial sensors, are integrated in the proposed system. As the main source for the localization, the GNSS technique suffers from Non-Line-Of-Sight (NLOS) propagation and multipath effects in urban canyons. This paper proposes to employ a novel GNSS positioning technique in the integration. The employed GNSS technique reduces the multipath and NLOS effects by using the 3D building map. In addition, the inertial sensor can describe the vehicle motion, but has a drift problem as time increases. This paper develops vision-based lane detection, which is firstly used for controlling the drift of the inertial sensor. Moreover, the lane keeping and changing behaviors are extracted from the lane detection function, and further reduce the lateral positioning error in the proposed localization system. We evaluate the integrated localization system in the challenging city urban scenario. The experiments demonstrate the proposed method has sub-meter accuracy with respect to mean positioning error.

NLOS Correction/Exclusion for GNSS Measurement Using RAIM and City Building Models.

Currently, global navigation satellite system (GNSS) receivers can provide accurate and reliable positioning service in open-field areas. However, their performance in the downtown areas of cities is still affected by the multipath and none-line-of-sight (NLOS) receptions. This paper proposes a new positioning method using 3D building models and the receiver autonomous integrity monitoring (RAIM) satellite selection method to achieve satisfactory positioning performance in urban area. The 3D building model uses a ray-tracing technique to simulate the line-of-sight (LOS) and NLOS signal travel distance, which is well-known as pseudorange, between the satellite and receiver. The proposed RAIM fault detection and exclusion (FDE) is able to compare the similarity between the raw pseudorange measurement and the simulated pseudorange. The measurement of the satellite will be excluded if the simulated and raw pseudoranges are inconsistent. Because of the assumption of the single reflection in the ray-tracing technique, an inconsistent case indicates it is a double or multiple reflected NLOS signal. According to the experimental results, the RAIM satellite selection technique can reduce by about 8.4% and 36.2% the positioning solutions with large errors (solutions estimated on the wrong side of the road) for the 3D building model method in the middle and deep urban canyon environment, respectively.

Inhibition of aggregation of amyloid ß42 by arginine-containing small compounds.

Aggregations of proteins are in many cases associated with neurodegenerative diseases such as Alzheimer's (AD). Small compounds capable of inhibiting protein aggregation are expected to be useful for not only in the treatment of disease but also in probing the structures of aggregated proteins. In previous studies using phage display, we found that arginine-rich short peptides consisting of four or seven amino acids bound to soluble 42-residue amyloid ß (Aß42) and inhibited globulomer (37/48 kDa oligomer) formation. In the present study, we searched for arginine-containing small molecules using the SciFinder searching service and tested their inhibitory activities against Aß42 aggregation, by sodium dodecyl sulfate (SDS)-PAGE and thioflavine T binding assay. Commercially available Arg-Arg-7-amino-4-trifluoromethylcoumarin was found to exhibit remarkable inhibitory activities to the formation of the globulomer and the fibril of Aß42. This chimera-type tri-peptide is expected to serve as the seed molecule of a potent inhibitor of the Aß aggregation process.

Identification of novel short peptide inhibitors of soluble 37/48 kDa oligomers of amyloid ß42.

Since the soluble oligomers of 42-residue amyloid ß (Aß42) might be neurotoxins at an early stage of Alzheimer's disease (AD), inhibition of soluble Aß42 oligomerization should be effective in the treatment of AD. We have found by phage display that a 7-residue peptide, SRPGLRR, exhibited inhibitory activity against soluble 37/48 kDa oligomers of Aß42. In the present study, we newly prepared 3- and 4-residue random peptides libraries and performed pannings of them against soluble Aß42 to search for important factors in the inhibition of Aß42 oligomerization. After the fifth round, arginine-containing peptides were enriched in both libraries. SDS-PAGE and size-exclusion chromatography indicated that the inhibitory activities of 4-residue peptides against the soluble 37/48 kDa oligomers of Aß42 were higher than those of the 3-residue peptides, and RFRK exhibited strong inhibitory activity as well as SRPGLRR. These short peptides should be useful for the suppression of soluble Aß42 oligomer formation.

Selection of peptide inhibitors of soluble Aß(1-42) oligomer formation by phage display.

There have been many reports suggesting that soluble oligomers of amyloid ß (Aß) are neurotoxins causing Alzheimer's disease (AD). Although inhibition of the soluble oligomerization of Aß is considered to be effective in the treatment of AD, almost all peptide inhibitors have been designed from the ß-sheet structure (H14-D23) of Aß(1-42). To obtain more potent peptides than the known inhibitors of the soluble-oligomer formation of Aß(1-42), we performed random screening by phage display. After fifth-round panning of a hepta-peptide library against soluble Aß(1-42), novel peptides containing arginine residues were enriched. These peptides were found to suppress specifically 37/48 kDa oligomer formation and to keep the monomeric form of Aß(1-42) even after 24 h of incubation, as disclosed by SDS-PAGE and size-exclusion chromatography. Thus we succeeded in acquiring novel efficient peptides for inhibition of soluble 37/48 kDa oligomer formation of Aß(1-42).

Analyses of conditions for KMSSS loop in tyrosyl-tRNA synthetase by building a mutant library.

The KMSKS motif is the ATP binding motif for aminoacylation process of class I aminoacyl-tRNA synthetases. Although researches based on natural proteins inform us about the contribution of natural amino acid sequences for the catalysis, they have difficulties in discussing the other alternative sequences and prohibited sequences for the motif to maintain the catalytic ability. In order to reveal such the conditions for the alternative and prohibited sequences, it is important to investigate a library of various mutants for the motif. For that purpose, we build a library of more than 200 mutants substituting the KMSSS loop, Lys204-Met205-Ser206-Ser207-Ser208, in tyrosyl-tRNA synthetase of Methanococcus jannaschii, and their catalytic abilities were examined by the Amber suppression method. Mutants of K204R and K204N still maintained catalytic abilities to a certain extent. On the other hand, a variety of alternative sequences for Ser206-Ser207-Ser208 were obtained, and some of those did not include either Ser or Thr, which were regarded as necessary residues in the KMSKS motif in previous works. In this article, catalytic activity of all the mutants are represented in detail and some suggestions for the condition of the motif are discussed.