CoFormerNet: A Transformer-Based Fusion Approach for Enhanced Vehicle-Infrastructure Cooperative Perception.

Vehicle-infrastructure cooperative perception is becoming increasingly crucial for autonomous driving systems and involves leveraging infrastructure's broader spatial perspective and computational resources. This paper introduces CoFormerNet, which is a novel framework for improving cooperative perception. CoFormerNet employs a consistent structure for both vehicle and infrastructure branches, integrating the temporal aggregation module and spatial-modulated cross-attention to fuse intermediate features at two distinct stages. This design effectively handles communication delays and spatial misalignment. Experimental results using the DAIR-V2X and V2XSet datasets demonstrated that CoFormerNet significantly outperformed the existing methods, achieving state-of-the-art performance in 3D object detection.

HP3D-V2V: High-Precision 3D Object Detection Vehicle-to-Vehicle Cooperative Perception Algorithm.

Cooperative perception in the field of connected autonomous vehicles (CAVs) aims to overcome the inherent limitations of single-vehicle perception systems, including long-range occlusion, low resolution, and susceptibility to weather interference. In this regard, we propose a high-precision 3D object detection V2V cooperative perception algorithm. The algorithm utilizes a voxel grid-based statistical filter to effectively denoise point cloud data to obtain clean and reliable data. In addition, we design a feature extraction network based on the fusion of voxels and PointPillars and encode it to generate BEV features, which solves the spatial feature interaction problem lacking in the PointPillars approach and enhances the semantic information of the extracted features. A maximum pooling technique is used to reduce the dimensionality and generate pseudoimages, thereby skipping complex 3D convolutional computation. To facilitate effective feature fusion, we design a feature level-based crossvehicle feature fusion module. Experimental validation is conducted using the OPV2V dataset to assess vehicle coperception performance and compare it with existing mainstream coperception algorithms. Ablation experiments are also carried out to confirm the contributions of this approach. Experimental results show that our architecture achieves lightweighting with a higher average precision (AP) than other existing models.

Enhanced Perception for Autonomous Vehicles at Obstructed Intersections: An Implementation of Vehicle to Infrastructure (V2I) Collaboration.

In urban intersections, the sensory capabilities of autonomous vehicles (AVs) are often hindered by visual obstructions, posing significant challenges to their robust and safe operation. This paper presents an implementation study focused on enhancing the safety and robustness of Connected Automated Vehicles (CAVs) in scenarios with occluded visibility at urban intersections. A novel LiDAR Infrastructure System is established for roadside sensing, combined with Baidu Apollo's Automated Driving System (ADS) and Cohda Wireless V2X communication hardware, and an integrated platform is established for roadside perception enhancement in autonomous driving. The field tests were conducted at the Singapore CETRAN (Centre of Excellence for Testing & Research of Autonomous Vehicles-NTU) autonomous vehicle test track, with the communication protocol adhering to SAE J2735 V2X communication standards. Communication latency and packet delivery ratio were analyzed as the evaluation metrics. The test results showed that the system can help CAV detect obstacles in advance under urban occluded scenarios.

Impact analysis of cooperative perception on the performance of automated driving in unsignalized roundabouts.

This paper reports the implementation and results of a simulation-based analysis of the impact of cloud/edge-enabled cooperative perception on the performance of automated driving in unsignalized roundabouts. This is achieved by comparing the performance of automated driving assisted by cooperative perception to that of a baseline system, where the automated vehicle relies only on its onboard sensing and perception for motion planning and control. The paper first provides the descriptions of the implemented simulation model, which integrates the SUMO road traffic generator and CARLA simulator. This includes descriptions of both the baseline and cooperative perception-assisted automated driving systems. We then define a set of relevant key performance indicators for traffic efficiency, safety, and ride comfort, as well as simulation scenarios to collect relevant data for our analysis. This is followed by the description of simulation scenarios, presentation of the results, and discussions of the insights learned from the results.

Object Detection Based on Roadside LiDAR for Cooperative Driving Automation: A Review.

Light Detection and Ranging (LiDAR) technology has the advantages of high detection accuracy, a wide range of perception, and not being affected by light. The 3D LiDAR is placed at the commanding height of the traffic scene, the overall situation can be grasped from the perspective of top view, and the trajectory of each object in the traffic scene can be accurately perceived in real time, and then the object information can be distributed to the surrounding vehicles or other roadside LiDAR through advanced wireless communication equipment, which can significantly improve the local perception ability of an autonomous vehicle. This paper first describes the characteristics of roadside LiDAR and the challenges of object detection and then reviews in detail the current methods of object detection based on a single roadside LiDAR and multi-LiDAR cooperatives. Then, some studies for roadside LiDAR perception in adverse weather and datasets released in recent years are introduced. Finally, some current open challenges and future works for roadside LiDAR perception are discussed. To the best of our knowledge, this is the first work to systematically study roadside LiDAR perception methods and datasets. It has an important guiding role in further promoting the research of roadside LiDAR perception for practical applications.

Cooperative Perception Technology of Autonomous Driving in the Internet of Vehicles Environment: A Review.

Cooperative perception, as a critical technology of intelligent connected vehicles, aims to use wireless communication technology to interact and fuse environmental information obtained by edge nodes with local perception information, which can improve vehicle perception accuracy, reduce latency, and eliminate perception blind spots. It has become a current research hotspot. Based on the analysis of the related literature on the Internet of vehicles (IoV), this paper summarizes the multi-sensor information fusion method, information sharing strategy, and communication technology of autonomous driving cooperative perception technology in the IoV environment. Firstly, cooperative perception information fusion methods, such as image fusion, point cloud fusion, and image-point cloud fusion, are summarized and compared according to the approaches of sensor information fusion. Secondly, recent research on communication technology and the sharing strategies of cooperative perception technology is summarized and analyzed in detail. Simultaneously, combined with the practical application of V2X, the influence of network communication performance on cooperative perception is analyzed, considering factors such as latency, packet loss rate, and channel congestion, and the existing research methods are discussed. Finally, based on the summary and analysis of the above studies, future research issues on cooperative perception are proposed, and the development trend of cooperative perception technology is forecast to help researchers in this field quickly understand the research status, hotspots, and prospects of cooperative perception technology.

Automated Driving with Cooperative Perception Based on CVFH and Millimeter-Wave V2I Communications for Safe and Efficient Passing through Intersections.

The development of automated driving is actively progressing, and connected cars are also under development. Connected cars are the technology of connecting vehicles to networks so that connected vehicles can enhance their services. Safety services are among the main services expected in connected car society. Cooperative perception belongs to safety services and improves safety by visualizing blind spots. This visualization is achieved by sharing sensor data via wireless communications. Therefore, the number of visualized blind spots highly depends upon the performance of wireless communications. In this paper, we analyzed the required sensor data rate to be shared for the cooperative perception in order to realize safe and reliable automated driving in an intersection scenario. The required sensor data rate was calculated by the combination of recognition and crossing decisions of an automated driving vehicle to adopt realistic assumptions. In this calculation, CVFH was used to derive tight requirements, and the minimum required braking aims to alleviate the traffic congestion around the intersection. At the end of the paper, we compare the required sensor data rate with the outage data rate realized by conventional and millimeter-wave communications, and show that millimeter-wave communications can support safe crossing at a realistic velocity.

ECPC-ICP: A 6D Vehicle Pose Estimation Method by Fusing the Roadside Lidar Point Cloud and Road Feature.

In the vehicle pose estimation task based on roadside Lidar in cooperative perception, the measurement distance, angle, and laser resolution directly affect the quality of the target point cloud. For incomplete and sparse point clouds, current methods are either less accurate in correspondences solved by local descriptors or not robust enough due to the reduction of effective boundary points. In response to the above weakness, this paper proposed a registration algorithm Environment Constraint Principal Component-Iterative Closest Point (ECPC-ICP), which integrated road information constraints. The road normal feature was extracted, and the principal component of the vehicle point cloud matrix under the road normal constraint was calculated as the initial pose result. Then, an accurate 6D pose was obtained through point-to-point ICP registration. According to the measurement characteristics of the roadside Lidars, this paper defined the point cloud sparseness description. The existing algorithms were tested on point cloud data with different sparseness. The simulated experimental results showed that the positioning MAE of ECPC-ICP was about 0.5% of the vehicle scale, the orientation MAE was about 0.26°, and the average registration success rate was 95.5%, which demonstrated an improvement in accuracy and robustness compared with current methods. In the real test environment, the positioning MAE was about 2.6% of the vehicle scale, and the average time cost was 53.19 ms, proving the accuracy and effectiveness of ECPC-ICP in practical applications.

Analysis of Cooperative Perception in Ant Traffic and Its Effects on Transportation System by Using a Congestion-Free Ant-Trail Model.

We investigated agent-based model simulations that mimic an ant transportation system to analyze the cooperative perception and communication in the system. On a trail, ants use cooperative perception through chemotaxis to maintain a constant average velocity irrespective of their density, thereby avoiding traffic jams. Using model simulations and approximate mathematical representations, we analyzed various aspects of the communication system and their effects on cooperative perception in ant traffic. Based on the analysis, insights about the cooperative perception of ants which facilitate decentralized self-organization is presented. We also present values of communication-parameters in ant traffic, where the system conveys traffic conditions to individual ants, which ants use to self-organize and avoid traffic-jams. The mathematical analysis also verifies our findings and provides a better understanding of various model parameters leading to model improvements.

Automated Driving with Cooperative Perception Using Millimeter-Wave V2V Communications for Safe Overtaking.

The combination of onboard sensors on vehicles with wireless communication has great advantages over the conventional driving systems in terms of safety and reliability. This technique is often called cooperative perception. Cooperative perception is expected to compensate for blind spots in dynamic maps, which are caused by obstacles. Few blind spots in dynamic maps can improve the safety and reliability of driving thanks to the additional information beyond the sensing of the onboard sensors. In this paper, we analyzed the required sensor data rate to be exchanged for the cooperative perception in order to enable a new level of safe and reliable automated driving in overtaking scenario. The required sensor data rate was calculated by the combination of recognition and vehicle movement to adopt realistic assumptions. In the end of the paper, we compared the required sensor data rate with the outage data rate realized by the conventional V2V communication and millimeter-wave communication. The results showed the indispensability of millimeter-wave communications in automated driving systems.

Standards-Compliant Multi-Protocol On-Board Unit for the Evaluation of Connected and Automated Mobility Services in Multi-Vendor Environments.

Vehicle-to-everything (V2X) communications enable real-time information exchange between vehicles and infrastructure, which extends the perception range of vehicles beyond the limits of on-board sensors and, thus, facilitating the realisation of cooperative, connected, and automated mobility (CCAM) services that will improve road safety and traffic efficiency. In the context of CCAM, the successful deployments of cooperative intelligent transport system (C-ITS) use cases, with the integration of advanced wireless communication technologies, are effectively leading to make transport safer and more efficient. However, the evaluation of multi-vendor and multi-protocol based CCAM service architectures can become challenging and complex. Additionally, conducting on-demand field trials of such architectures with real vehicles involved is prohibitively expensive and time-consuming. In order to overcome these obstacles, in this paper, we present the development of a standards-compliant experimental vehicular on-board unit (OBU) that supports the integration of multiple V2X protocols from different vendors to communicate with heterogeneous cloud-based services that are offered by several original equipment manufacturers (OEMs). We experimentally demonstrate the functionalities of the OBU in a real-world deployment of a cooperative collision avoidance service infrastructure that is based on edge and cloud servers. In addition, we measure end-to-end application-level latencies of multi-protocol supported V2X information flows to show the effectiveness of interoperability in V2X communications between different vehicle OEMs.

Driving Environment Perception Based on the Fusion of Vehicular Wireless Communications and Automotive Remote Sensors.

Driving environment perception for automated vehicles is typically achieved by the use of automotive remote sensors such as radars and cameras. A vehicular wireless communication system can be viewed as a new type of remote sensor that plays a central role in connected and automated vehicles (CAVs), which are capable of sharing information with each other and also with the surrounding infrastructure. In this paper, we present the design and implementation of driving environment perception based on the fusion of vehicular wireless communications and automotive remote sensors. A track-to-track fusion of high-level sensor data and vehicular wireless communication data was performed to accurately and reliably locate the remote target in the vehicle surroundings and predict the future trajectory. The proposed approach was implemented and evaluated in vehicle tests conducted at a proving ground. The experimental results demonstrate that using vehicular wireless communications in conjunction with the on-board sensors enables improved perception of the surrounding vehicle located at varying longitudinal and lateral distances. The results also indicate that vehicle future trajectory and potential crash involvement can be reliably predicted with the proposed system in different cut-in driving scenarios.

Research on Cooperative Perception of MUSVs in Complex Ocean Conditions.

Since the working environment of Multiple Unmanned Surface Vehicles (MUSVs) is accompanied by a large number of uncertainties and various hazards, in order to ensure the collision avoidance capability of MUSVs in complex marine environments, the perception of complex marine environments by MUSVs is the first problem that needs to be solved. A cooperative perception framework with uncertain event detection, cooperative collision avoidance pattern recognition and environmental ontology model is proposed to realize the cooperative perception process of MUSVs using ontology and Bayesian network theory. The cooperative perception approach was validated by simulating experiments. Results show the effectiveness of cooperative perception approach.

A Grid-Based Framework for Collective Perception in Autonomous Vehicles.

Today, perception solutions for Automated Vehicles rely on sensors on board the vehicle, which are limited by the line of sight and occlusions caused by any other elements on the road. As an alternative, Vehicle-to-Everything (V2X) communications allow vehicles to cooperate and enhance their perception capabilities. Besides announcing its own presence and intentions, services such as Collective Perception (CPS) aim to share information about perceived objects as a high-level description. This work proposes a perception framework for fusing information from on-board sensors and data received via CPS messages (CPM). To that end, the environment is modeled using an occupancy grid where occupied, and free and uncertain space is considered. For each sensor, including V2X, independent grids are calculated from sensor measurements and uncertainties and then fused in terms of both occupancy and confidence. Moreover, the implementation of a Particle Filter allows the evolution of cell occupancy from one step to the next, allowing for object tracking. The proposed framework was validated on a set of experiments using real vehicles and infrastructure sensors for sensing static and dynamic objects. Results showed a good performance even under important uncertainties and delays, hence validating the viability of the proposed framework for Collective Perception.

Multi-Vehicle Cooperative Target Tracking with Time-Varying Localization Uncertainty via Recursive Variational Bayesian Inference.

Cooperative target tracking by multiple vehicles connected through inter-vehicle communication is a promising way to improve the estimation of target state. The effectiveness of cooperative tracking closely depends on the accuracy of relative localization between host and cooperative vehicles. However, the localization signal usually provided by the satellite-based navigation system is rather susceptible to dynamic driving environment, thus influencing the effectiveness of cooperative tracking. In order to implement reliable cooperative tracking, especially when the statistical characteristic of the relative localization noise is time-varying and uncertain, this paper presents a recursive Bayesian framework which jointly estimates the state of the target and the cooperative vehicle as well as the localization noise parameter. An online variational Bayesian inference algorithm is further developed to achieve efficient recursive estimate. The simulation results verify that our proposed algorithm can effectively boost the accuracy of target tracking when the localization noise dynamically changes over time.

Semantic Distributed Data for Vehicular Networks Using the Inter-Planetary File System.

Vehicular networks provide means to distribute data among intelligent vehicles, increasing their efficiency and the safety of their occupants. While connected to these networks, vehicles have access to various kinds of information shared by other vehicles and road-side units (RSUs). This information includes helpful resources, such as traffic state or remote sensors. An efficient and fast system to get access to this information is important but unproductive if the data are not appropriately structured, accessible, and easy to process. This paper proposes the creation of a semantic distributed network using content-addressed networking and peer-to-peer (P2P) connections. In this open and collaborative network, RSUs and vehicles use ontologies to semantically represent information and facilitate the development of intelligent autonomous agents capable of navigating and processing the shared data. In order to create this P2P network, this paper makes use of the Inter-Planetary File System (IPFS), an open source solution that provides secure, reliable, and efficient content-addressed distributed storage over standard IP networks using the new QUIC protocol. This paper highlights the feasibility of this proposal and compares it with the state-of-the-art. Results show that IPFS is a promising technology that offers a great balance between functionality, performance, and security.

Networked Roadside Perception Units for Autonomous Driving.

Vehicle-to-Everything (V2X) communication enhances the capability of autonomous driving through better safety, efficiency, and comfort. In particular, sensor data sharing, known as cooperative perception, is a crucial technique to accommodate vulnerable road users in a cooperative intelligent transport system (ITS). In this paper, we describe a roadside perception unit (RSPU) that combines sensors and roadside units (RSUs) for infrastructure-based cooperative perception. We propose a software called AutoC2X that we designed to realize cooperative perception for RSPUs and vehicles. We also propose the concept of networked RSPUs, which is the inter-connection of RSPUs along a road over a wired network, and helps realize broader cooperative perception. We evaluated the RSPU system and the networked RSPUs through a field test, numerical analysis, and simulation experiments. Field evaluation showed that, even in the worst case, our RSPU system can deliver messages to an autonomous vehicle within 100 ms. The simulation result shows that the proposed priority algorithm achieves a wide perception range with a high delivery ratio and low latency, especially under heavy road traffic conditions.

Robust Cooperative Multi-Vehicle Tracking with Inaccurate Self-Localization Based on On-Board Sensors and Inter-Vehicle Communication.

The fusion of on-board sensors and transmitted information via inter-vehicle communication has been proved to be an effective way to increase the perception accuracy and extend the perception range of connected intelligent vehicles. The current approaches rely heavily on the accurate self-localization of both host and cooperative vehicles. However, such information is not always available or accurate enough for effective cooperative sensing. In this paper, we propose a robust cooperative multi-vehicle tracking framework suitable for the situation where the self-localization information is inaccurate. Our framework consists of three stages. First, each vehicle perceives its surrounding environment based on the on-board sensors and exchanges the local tracks through inter-vehicle communication. Then, an algorithm based on Bayes inference is developed to match the tracks from host and cooperative vehicles and simultaneously optimize the relative pose. Finally, the tracks associated with the same target are fused by fast covariance intersection based on information theory. The simulation results based on both synthesized data and a high-quality physics-based platform show that our approach successfully implements cooperative tracking without the assistance of accurate self-localization.

Collective Perception: A Safety Perspective.

Vehicle-to-everything (V2X) communication is seen as one of the main enabling technologies for automated vehicles. Collective perception is especially promising, as it allows connected traffic participants to "see through the eyes of others" by sharing sensor-detected objects via V2X communication. Its benefit is typically assessed in terms of the increased object update rate, redundancy, and awareness. To determine the safety improvement thanks to collective perception, the authors introduce new metrics, which quantify the environmental risk awareness of the traffic participants. The performance of the V2X service is then analyzed with the help of the test platform TEPLITS, using real traffic traces from German highways, amounting to over 100 h of total driving time. The results in the considered scenarios clearly show that collective perception not only contributes to the accuracy and integrity of the vehicles' environmental perception, but also that a V2X market penetration of at least 25% is necessary to increase traffic safety from a "risk of serious traffic accidents" to a "residual hypothetical risk of collisions without minor injuries" for traffic participants equipped with non-redundant 360° sensor systems. These results support the ongoing worldwide standardization efforts of the collective perception service.

Demonstrations of Cooperative Perception: Safety and Robustness in Connected and Automated Vehicle Operations.

Cooperative perception, or collective perception (CP), is an emerging and promising technology for intelligent transportation systems (ITS). It enables an ITS station (ITS-S) to share its local perception information with others by means of vehicle-to-X (V2X) communication, thereby achieving improved efficiency and safety in road transportation. In this paper, we present our recent progress on the development of a connected and automated vehicle (CAV) and intelligent roadside unit (IRSU). The main contribution of the work lies in investigating and demonstrating the use of CP service within intelligent infrastructure to improve awareness of vulnerable road users (VRU) and thus safety for CAVs in various traffic scenarios. We demonstrate in experiments that a connected vehicle (CV) can "see" a pedestrian around the corners. More importantly, we demonstrate how CAVs can autonomously and safely interact with walking and running pedestrians, relying only on the CP information from the IRSU through vehicle-to-infrastructure (V2I) communication. This is one of the first demonstrations of urban vehicle automation using only CP information. We also address in the paper the handling of collective perception messages (CPMs) received from the IRSU, and passing them through a pipeline of CP information coordinate transformation with uncertainty, multiple road user tracking, and eventually path planning/decision-making within the CAV. The experimental results were obtained with manually driven CV, fully autonomous CAV, and an IRSU retrofitted with vision and laser sensors and a road user tracking system.