A New 3D Object Pose Detection Method Using LIDAR Shape Set.

In object detection systems for autonomous driving, LIDAR sensors provide very useful information. However, problems occur because the object representation is greatly distorted by changes in distance. To solve this problem, we propose a LIDAR shape set that reconstructs the shape surrounding the object more clearly by using the LIDAR point information projected on the object. The LIDAR shape set restores object shape edges from a bird's eye view by filtering LIDAR points projected on a 2D pixel-based front view. In this study, we use this shape set for two purposes. The first is to supplement the shape set with a LIDAR Feature map, and the second is to divide the entire shape set according to the gradient of the depth and density to create a 2D and 3D bounding box proposal for each object. We present a multimodal fusion framework that classifies objects and restores the 3D pose of each object using enhanced feature maps and shape-based proposals. The network structure consists of a VGG -based object classifier that receives multiple inputs and a LIDAR-based Region Proposal Networks (RPN) that identifies object poses. It works in a very intuitive and efficient manner and can be extended to other classes other than vehicles. Our research has outperformed object classification accuracy (Average Precision, AP) and 3D pose restoration accuracy (3D bounding box recall rate) based on the latest studies conducted with KITTI data sets.

Prediction of crime occurrence from multi-modal data using deep learning.

In recent years, various studies have been conducted on the prediction of crime occurrences. This predictive capability is intended to assist in crime prevention by facilitating effective implementation of police patrols. Previous studies have used data from multiple domains such as demographics, economics, and education. Their prediction models treat data from different domains equally. These methods have problems in crime occurrence prediction, such as difficulty in discovering highly nonlinear relationships, redundancies, and dependencies between multiple datasets. In order to enhance crime prediction models, we consider environmental context information, such as broken windows theory and crime prevention through environmental design. In this paper, we propose a feature-level data fusion method with environmental context based on a deep neural network (DNN). Our dataset consists of data collected from various online databases of crime statistics, demographic and meteorological data, and images in Chicago, Illinois. Prior to generating training data, we select crime-related data by conducting statistical analyses. Finally, we train our DNN, which consists of the following four kinds of layers: spatial, temporal, environmental context, and joint feature representation layers. Coupled with crucial data extracted from various domains, our fusion DNN is a product of an efficient decision-making process that statistically analyzes data redundancy. Experimental performance results show that our DNN model is more accurate in predicting crime occurrence than other prediction models.

Structural alterations in the prefrontal cortex mediate the relationship between Internet gaming disorder and depressed mood.

Adaptive gaming use has positive effects, whereas depression has been reported to be prevalent in Internet gaming disorder (IGD). However, the neural correlates underlying the association between depression and Internet gaming remain unclear. Moreover, the neuroanatomical profile of the striatum in IGD is relatively less clear despite its important role in addiction. We found lower gray matter (GM) density in the left dorsolateral prefrontal cortex (DLPFC) in the IGD group than in the Internet gaming control (IGC) group and non-gaming control (NGC) group, and the GM density was associated with lifetime usage of Internet gaming, depressed mood, craving, and impulsivity in the gaming users. Striatal volumetric analysis detected a significant reduction in the right nucleus accumbens (NAcc) in the IGD group and its association with lifetime usage of gaming and depression. These findings suggest that alterations in the brain structures involved in the reward system are associated with IGD-related behavioral characteristics. Furthermore, the DLPFC, involved in cognitive control, was observed to serve as a mediator in the association between prolonged gaming and depressed mood. This finding may provide insight into an intervention strategy for treating IGD with comorbid depression.

Multiple Objects Fusion Tracker Using a Matching Network for Adaptively Represented Instance Pairs.

Multiple-object tracking is affected by various sources of distortion, such as occlusion, illumination variations and motion changes. Overcoming these distortions by tracking on RGB frames, such as shifting, has limitations because of material distortions caused by RGB frames. To overcome these distortions, we propose a multiple-object fusion tracker (MOFT), which uses a combination of 3D point clouds and corresponding RGB frames. The MOFT uses a matching function initialized on large-scale external sequences to determine which candidates in the current frame match with the target object in the previous frame. After conducting tracking on a few frames, the initialized matching function is fine-tuned according to the appearance models of target objects. The fine-tuning process of the matching function is constructed as a structured form with diverse matching function branches. In general multiple object tracking situations, scale variations for a scene occur depending on the distance between the target objects and the sensors. If the target objects in various scales are equally represented with the same strategy, information losses will occur for any representation of the target objects. In this paper, the output map of the convolutional layer obtained from a pre-trained convolutional neural network is used to adaptively represent instances without information loss. In addition, MOFT fuses the tracking results obtained from each modality at the decision level to compensate the tracking failures of each modality using basic belief assignment, rather than fusing modalities by selectively using the features of each modality. Experimental results indicate that the proposed tracker provides state-of-the-art performance considering multiple objects tracking (MOT) and KITTIbenchmarks.

Object Detection and Classification by Decision-Level Fusion for Intelligent Vehicle Systems.

To understand driving environments effectively, it is important to achieve accurate detection and classification of objects detected by sensor-based intelligent vehicle systems, which are significantly important tasks. Object detection is performed for the localization of objects, whereas object classification recognizes object classes from detected object regions. For accurate object detection and classification, fusing multiple sensor information into a key component of the representation and perception processes is necessary. In this paper, we propose a new object-detection and classification method using decision-level fusion. We fuse the classification outputs from independent unary classifiers, such as 3D point clouds and image data using a convolutional neural network (CNN). The unary classifiers for the two sensors are the CNN with five layers, which use more than two pre-trained convolutional layers to consider local to global features as data representation. To represent data using convolutional layers, we apply region of interest (ROI) pooling to the outputs of each layer on the object candidate regions generated using object proposal generation to realize color flattening and semantic grouping for charge-coupled device and Light Detection And Ranging (LiDAR) sensors. We evaluate our proposed method on a KITTI benchmark dataset to detect and classify three object classes: cars, pedestrians and cyclists. The evaluation results show that the proposed method achieves better performance than the previous methods. Our proposed method extracted approximately 500 proposals on a 1226 × 370 image, whereas the original selective search method extracted approximately 10 6 × n proposals. We obtained classification performance with 77.72% mean average precision over the entirety of the classes in the moderate detection level of the KITTI benchmark dataset.

An Analysis of the Effects of Smartphone Push Notifications on Task Performance with regard to Smartphone Overuse Using ERP.

Smartphones are used ubiquitously worldwide and are essential tools in modern society. However, smartphone overuse is an emerging social issue, and limited studies have objectively assessed this matter. The majority of previous studies have included surveys or behavioral observation studies. Since a previous study demonstrated an association between increased push notifications and smartphone overuse, we investigated the effects of push notifications on task performance. We detected changes in brainwaves generated by smartphone push notifications using the N200 and P300 components of event-related potential (ERP) to investigate both concentration and cognitive ability. ERP assessment indicated that, in both risk and nonrisk groups, the lowest N200 amplitude and the longest latency during task performance were found when push notifications were delivered. Compared to the nonrisk group, the risk group demonstrated lower P300 amplitudes and longer latencies. In addition, the risk group featured a higher rate of error in the Go-Nogo task, due to the negative influence of smartphone push notifications on performance in both risk and nonrisk groups. Furthermore, push notifications affected subsequent performance in the risk group.

Changes in ocular parameters depending on the motion-in-depth of a three-dimensional display.

OBJECTIVE: To compare changes in ocular parameters after watching a display of three-dimensional (3D) images, with reference to motion-in-depth and viewer age. METHODS: A total of 30 healthy subjects were enrolled (of whom 17 were aged 20-30 years and 13, 40-50 years). All subjects watched 3D displays with binocular disparities of 1° or 3° run towards the viewer (who wore polarised glasses) at two defined velocities (slow, 105 mm/s; fast, 257 mm/s) for 15 min at intervals of 1 week. The viewing distance was 1.020 m. The near point of accommodation (NPA) and near point of convergence (NPC), and the tear break-up time (tBUT) of each subject, were measured before and after watching the 3D display. All parameters were repeatedly measured at intervals of 10 min after watching until baseline values became re-established. RESULTS: NPA and NPC deteriorated more, and tBUT decreased more, after watching a 3D display with fast rather than slow motion-in-depth (all p values <0.05). NPA deteriorated more in those aged 40-50 years as compared in those aged 20-30 years after watching a display of binocular disparity of 3° at fast motion-in-depth (p=0.028). NPC deteriorated more in those aged 40-50 years as compared in those aged 20-30 years after watching a display of binocular disparity of 3° at slow and fast motion-in-depth (p=0.001). The NPA and NPC recovery times were longer after watching at fast motion-in-depth than slow motion-in-depth (p<0.05). The decrease of tBUT was greater after watching at fast rather than slow motion-in-depth but only when the binocular disparity was 1°. All parameters returned to baseline values within 80 min. CONCLUSIONS: Motion-in-depth has an important influence on ocular parameters when a 3D display is watched, and our information would provide some basis in manufacturing 3D equipment.

Changes in ocular factors according to depth variation and viewer age after watching a three-dimensional display.

OBJECTIVE: To investigate changes in ocular factors according to the binocular disparity in three-dimensional (3D) images and age after watching 3D display. METHODS: A total of 38 volunteers were enrolled, and they watched a 3D display with a 1° or 3° disparity for 30 min at an interval of 1 week. The near point of accommodation (NPA), near point of convergence (NPC) and tear break-up time (tBUT) of each subject were measured before and after watching the 3D display. In addition, the tear meniscus height and depth were measured using Visante optical coherence tomography and tear osmolarity was measured using TearLab osmometer. A survey of subjective symptoms was also conducted. RESULTS: NPA and NPC increased after watching the 3D display (p<0.05). NPC and NPA increased more in the 40s-50s group (i.e., subjects aged in their 40s and 50s) than in the 20s-30s group (ie, subjects aged in their 20s and 30s) after watching 3D content with a 3° disparity (p<0.05). tBUT and tear meniscus height and depth decreased after watching 3D content (p<0.05). They decreased more in the 40s-50s group than in the 20s-30s group after watching 3D content with a 3° disparity (p<0.05). Recovery times of NPA and NPC were significantly greater after watching 3D content with a 3° disparity and in the 40s-50s group (p<0.05). CONCLUSIONS: Watching a 3D display affects accommodation and convergence abilities and tear dynamics in a transient fashion, especially in the case of 3D images with a large binocular disparity, and in older subjects. These results provide helpful information for establishment of guidelines for 3D equipment manufacturers.