ABSTRACT
BACKGROUND: Measuring variation in perioperative outcomes to accurately discriminate performance between surgical providers may be limited by reliability. We aimed to evaluate reliability estimates of metrics associated with lung cancer resection. METHODS: We performed a retrospective cohort study utilizing the 2015 National Cancer Database to identify patients undergoing lung cancer resection. Primary outcomes were reliability estimates for perioperative outcomes and for measures of adherence to clinical benchmarks, generated through hierarchical multilevel modeling techniques. RESULTS: We identified 27,300 patients undergoing resection. Overall risk-adjusted and reliability-adjusted 30-day and 90-day mortality rates were 1.7% and 3.3%, respectively; 61.0% and 41.1% of eligible patients received stage-appropriate adjuvant and neoadjuvant therapy. Video-assisted thoracoscopic surgery was performed in 59.6% of cases with clinical stage I disease. The mean reliability of 30-day and 90-day mortality was 0.11 ± 0.09 and 0.22 ± 0.15, respectively; for performing video-assisted thoracoscopic surgery for stage I disease, reliability was 0.97 ± 0.04. When stratified by hospital volume quartile, the mean reliability of 30-day mortality was 0.04 ± 0.03 in the lowest quartile and 0.20 ± 0.10 in the highest quartile. Only 14% of hospitals met an established 0.7 reliability benchmark for 30-day and 90-day mortality, but over 97% of hospitals exceeded these benchmarks for providing stage-appropriate systemic therapy and performing VATS for stage I disease. CONCLUSIONS: Metrics used to compare lung cancer surgical performance between providers have varying levels of reliability. Reliability should be considered when profiling providers, which will become particularly important as lung cancer treatment under screening programs continues to expand.
