Inaccurate Risk Assessment by the ACS NSQIP Risk Calculator in Aortic Surgery.

OBJECTIVES: The aim of this retrospective study was to assess the predictive performance of the American College of Surgeons (ACS) risk calculator for aortic aneurysm repair for the patient population of a Dutch tertiary referral hospital. METHODS: This retrospective study included all patients who underwent elective endovascular or open aortic aneurysm repair at our institution between the years 2013 and 2019. Preoperative patient demographics and postoperative complication data were collected, and individual risk assessments were generated using five different current procedural terminology (CPT) codes. Receiver operating characteristic (ROC) curves, calibration plots, Brier scores, and Index of Prediction Accuracy (IPA) values were generated to evaluate the predictive performance of the ACS risk calculator in terms of discrimination and calibration. RESULTS: Two hundred thirty-four patients who underwent elective endovascular or open aortic aneurysm repair were identified. Only five out of thirteen risk predictions were found to be sufficiently discriminative. Furthermore, the ACS risk calculator showed a structurally insufficient calibration. Most Brier scores were close to 0; however, comparison to a null model though IPA-scores showed the predictions generated by the ACS risk calculator to be inaccurate. Overall, the ACS risk calculator showed a consistent underestimation of the risk of complications. CONCLUSIONS: The ACS risk calculator proved to be inaccurate within the framework of endovascular and open aortic aneurysm repair in our medical center. To minimize the effects of patient selection and cultural differences, multicenter collaboration is necessary to assess the performance of the ACS risk calculator in aortic surgery.

Risk assessment in aortic aneurysm repair by medical specialists versus the American College of Surgeons National Surgical Quality Improvement Program risk calculator outcomes.

OBJECTIVE: The aim of this online clinical vignette-based survey study was to compare risk assessments by vascular surgeons, anaesthesiologists and interventional radiologists involved in treating patients with aortic aneurysms in the Netherlands with the NSQIP risk calculator outcomes. METHODS: Participants, recruited using purposive sampling, provided their estimation of the likelihood of postoperative complications and events following aortic surgery in five fictional cases. These cases were subsequently scored using the NSQIP calculator. The risk assessments were statistically analysed using the ANOVA and student t-test. RESULTS: All participating specialists i.e. twelve vascular surgeons, ten interventional radiologists and ten anaesthesiologists completed the survey. In the vast majority of outcomes and vignettes, no significant differences were found between various specialists, whereas significant differences were found between the NSQIP risk calculator outcomes and the combined risk assessments of the specialists. Overall, specialist risk assessments differ from the NSQIP, but neither particularly higher nor lower compared to the risk calculator. CONCLUSIONS: Risk assessment by vascular surgeons, anaesthesiologists and interventional radiologists differs significantly with NSQIP risk calculator outcomes, within the framework of both endovascular and open aortic aneurysm repair. Based on these results, implementing the NSQIP risk calculator in preoperative workup could be of added value in both patient planning as well as adequately informing patients for obtaining consent.