Examining the performance of engineering treatment evaluation methodologies using the hypothetical treatment and actual treatment settings.

The selection of treatment evaluation methodology is paramount in determining reliable crash modification factors (CMFs) for engineering treatments. A lack of ground truth makes it cumbersome to examine the performance of treatment evaluation methodologies. In addition, a sound methodological framework is critical for evaluating the performances of treatment evaluation methodologies. In addressing these challenges, this study proposed a framework for assessing treatment evaluation methodologies by hypothetical treatments with known ground truth and actual real-world treatments. In particular, this study examined three before-after treatment evaluation approaches: 1) Empirical Bayes, 2) Simulation-based Empirical Bayes, and 3) Full Bayes methods. In addition, this study examined the Cross-Sectional treatment evaluation methodology. The methodological framework utilized five datasets of hypothetical treatment with known ground truth based on the hotspot identification method and a real-world dataset of wide centerline treatment on two-lane, two-way rural highways in Queensland, Australia. Results showed that all the methods could identify the ground truth of hypothetical treatments, but the Full Bayes approach better predicts the known ground truth compared to Empirical Bayes, Simulation-based Empirical Bayes, and Cross-Sectional methods. The Full Bayes approach was also found to provide the most precise estimate for real-world wide centerline treatment along rural highways compared to other methods. Moreover, the current study highlighted that the Cross-Sectional method offers a viable estimate of treatment effectiveness in case the before-period data is limited.

Influence of segmentation approaches on the before-after evaluation of engineering treatments: A hypothetical treatment approach.

The segmentation of highways is a fundamental step in estimating crash frequency models and conducting a before-after evaluation of engineering treatments, but the effects of segmentation approaches on the engineering treatment evaluations are not known very well. This study examined the effects of segmentation approaches on the before-after evaluation of engineering treatments. In particular, this study evaluated four segmentation approaches by applying the Empirical Bayes technique to a dataset for which the ground truth was known. Four segmentation approaches included Highway Safety Manual (HSM), Fixed (kilometre post), Fisher's, and K-means segmentation. This study utilized a 440 km stretch of rural two-lane two-way highway in Queensland, Australia, to prepare a dataset with known ground truth. The treatment under evaluation was a hypothetical treatment, which should yield a crash modification factor (CMF) of 1. For assigning hypothetical treatment, a total of fifteen datasets were prepared, including ten datasets based on the random assignment and five datasets based on the hotspot identification method. Following the before-after evaluation using the Empirical Bayes technique, the results showed that HSM and Fixed segmentation approaches predict the ground truth in both dataset types. From random assignment datasets, the estimated CMFs using HSM, Fixed, Fisher's, and K-means segmentation approaches deviated from the true CMF (i.e., 1) by 2.32 %, 5.30 %, 6.08 %, and 8.62 %, respectively. In the case of hotspots, the corresponding deviations of CMFs were 8.57 %, 9.37 %, 28.84 %, and 35.43 %, respectively. Overall, HSM segmentation best identified the actual treatment effect, followed by the Fixed segmentation. If the variables to define homogeneity for HSM segmentation are limited, then Fixed segmentation can yield reliable crash modification factors from the before-after treatment evaluations than the crash-based segmentation approaches.

A simulation-based empirical bayes approach: Incorporating unobserved heterogeneity in the before-after evaluation of engineering treatments.

The Empirical Bayes approach for before-after evaluation methodology utilizing the negative binomial model does not account well for unobserved heterogeneity. Building on the Empirical Bayes approach, the objective of this study was to propose a framework to accommodate unobserved heterogeneity in before-after countermeasure evaluation. In particular, this study has proposed a simulation-based Empirical Bayes approach by applying the panel random parameters negative binomial model with parameterized overdispersion (PRNB-PO) to evaluate the effectiveness of engineering treatments. The proposed framework has been tested for the wide centerline treatment (WCLT) on rural two-lane two-way highways in Australia. The empirical analysis included 511 km of WCLT treated highways in a before-after evaluation within a time period of 2010 - 2018 and 430 km of reference sites in Queensland, Australia. The PRNB-PO models outperformed the traditional negative binomial models in terms of goodness-of-fit and prediction performance for total injury crashes, and fatal and serious injury (FSI) crashes. The simulation-based Empirical Bayes approach using the PRNB-PO model resulted in more precise estimates of crash modification factors than the standard Empirical Bayes approach. The WCLT is found to result in significant reductions in total injury crashes by 28.21% (95% confidence interval (CI) = 22.92 - 33.50%), FSI crashes by 13.90% (95% CI = 6.99 - 20.81%), and head-on crashes by 25.45% (95% CI = 14.87 - 36.03%). Overall, WCLT is an effective engineering treatment and should be considered a low-cost countermeasure on rural two-lane two-way highways.