PRISMA: A novel approach for deriving probabilistic surrogate safety measures for risk evaluation.

Surrogate Safety Measures (SSMs) are used to express road safety in terms of the safety risk in traffic conflicts. Typically, SSMs rely on assumptions regarding the future evolution of traffic participant trajectories to generate a measure of risk, restricting their applicability to scenarios where these assumptions are valid. In response to this limitation, we present the novel Probabilistic RISk Measure derivAtion (PRISMA) method. The objective of the PRISMA method is to derive SSMs that can be used to calculate in real time the probability of a specific event (e.g., a crash). The PRISMA method adopts a data-driven approach to predict the possible future traffic participant trajectories, thereby reducing the reliance on specific assumptions regarding these trajectories. Since the PRISMA is not bound to specific assumptions, the PRISMA method offers the ability to derive multiple SSMs for various scenarios. The occurrence probability of the specified event is based on simulations and combined with a regression model, this enables our derived SSMs to make real-time risk estimations. To illustrate the PRISMA method, an SSM is derived for risk evaluation during longitudinal traffic interactions. Since there is no known method to objectively estimate risk from first principles, i.e., there is no known risk ground truth, it is very difficult, if not impossible, to objectively compare the relative merits of two SSMs. Instead, we provide a method for benchmarking our derived SSM with respect to expected risk trends. The application of the benchmarking illustrates that the SSM matches the expected risk trends. Whereas the derived SSM shows the potential of the PRISMA method, future work involves applying the approach for other types of traffic conflicts, such as lateral traffic conflicts or interactions with vulnerable road users.

How certain are we that our automated driving system is safe?

OBJECTIVE: Regulations are currently being drafted by the European Commission for the safe introduction of automated driving systems (ADSs) with conditional or higher automation (SAE level 3 and above). One of the main challenges for complying with the drafted regulations is proving that the residual risk of an ADS is lower than the existing state of the art without the ADS and that the current safety state of European roads is not compromised. Therefore, much research has been conducted to estimate the safety risk of ADS. One proposed method for estimating the risk is data-driven, scenario-based assessment, where tests are partially automatically generated based on recorded traffic data. Although this is a promising method, uncertainties in the estimated risk arise from, among others, the limited number of tests that are conducted and the limited data that have been used to generate the tests. This work addresses the following question: "Given the limitations of the data and the number of tests, what is the uncertainty of the estimated safety risk of the ADS?" METHODS: To compute the safety risk, parameterized test scenarios are based on large-scale collections of road scenarios that are stored in a scenario database. The exposure of the scenarios and the parameter distributions are estimated using the data as well as confidence bounds of these estimates. Next, virtual simulations are conducted of the scenarios for a variety of parameter values. Using a probabilistic framework, all results are combined to estimate the residual risk as well as the uncertainty of this estimation. RESULTS: The results are used to provide confidence bounds on the calculated fatality rate in case an ADS is implemented in the vehicle. For example, using the proposed probabilistic framework, it is possible to claim with 95% certainty that the fatality rate is less than 10-7 fatalities per hour of driving. The proposed method is illustrated with a case study in which the risk and its uncertainty are quantified for a longitudinal controller in 3 different types of scenarios. The case study code is publicly available. CONCLUSIONS: If results show that the uncertainty is too high, the proposed method allows answering questions like "How much more data do we need?" or "How many more (virtual) simulations must be conducted?" Therefore, the method can be used to set requirements on the amount of data and the number of (virtual) simulations. For a reliable risk estimate, though, much more data are needed than those used in the case study. Furthermore, because the method relies on (virtual) simulations, the reliability of the result depends on the validity of the models used in the simulations. The presented case study illustrates that the proposed method is able to quantify the uncertainty of the estimated safety risk of an ADS. Future work involves incorporating the proposed method into the type approval framework for future ADSs of SAE levels 3, 4, and 5, as proposed in the upcoming European Union implementing regulation for ADS.

Safety assessment of automated vehicles: how to determine whether we have collected enough field data?

Objective: The amount of collected field data from naturalistic driving studies is quickly increasing. The data are used for, among others, developing automated driving technologies (such as crash avoidance systems), studying driver interaction with such technologies, and gaining insights into the variety of scenarios in real-world traffic. Because data collection is time consuming and requires high investments and resources, questions like "Do we have enough data?," "How much more information can we gain when obtaining more data?," and "How far are we from obtaining completeness?" are highly relevant. In fact, deducing safety claims based on collected data-for example, through testing scenarios based on collected data-requires knowledge about the degree of completeness of the data used. We propose a method for quantifying the completeness of the so-called activities in a data set. This enables us to partly answer the aforementioned questions. Method: In this article, the (traffic) data are interpreted as a sequence of different so-called scenarios that can be grouped into a finite set of scenario classes. The building blocks of scenarios are the activities. For every activity, there exists a parameterization that encodes all information in the data of each recorded activity. For each type of activity, we estimate a probability density function (pdf) of the associated parameters. Our proposed method quantifies the degree of completeness of a data set using the estimated pdfs. Results: To illustrate the proposed method, 2 different case studies are presented. First, a case study with an artificial data set, of which the underlying pdfs are known, is carried out to illustrate that the proposed method correctly quantifies the completeness of the activities. Next, a case study with real-world data is performed to quantify the degree of completeness of the acquired data for which the true pdfs are unknown. Conclusion: The presented case studies illustrate that the proposed method is able to quantify the degree of completeness of a small set of field data and can be used to deduce whether sufficient data have been collected for the purpose of the field study. Future work will focus on applying the proposed method to larger data sets. The proposed method will be used to evaluate the level of completeness of the data collection on Singaporean roads, aimed at defining relevant test cases for the autonomous vehicle road approval procedure that is being developed in Singapore.