Can Machine Learning Methods Produce Accurate and Easy-to-Use Preoperative Prediction Models of One-Year Improvements in Pain and Functioning After Knee Arthroplasty?

BACKGROUND: Approximately 15%-20% of total knee arthroplasty (TKA) patients do not experience clinically meaningful improvements. We sought to compare the accuracy and parsimony of several machine learning strategies for developing predictive models of failing to experience minimal clinically important differences in patient-reported outcome measures (PROMs) 1 year after TKA. METHODS: Patients (N = 587) in 3 large Veteran Health Administration facilities completed PROMs before and 1 year after TKA (92% follow-up). Preoperative PROMs and electronic health record data were used to develop and validate models to predict failing to experience at least a minimal clinically important difference in Knee Injury and Osteoarthritis Outcome Score (KOOS) Total, KOOS JR, and KOOS subscales (Pain, Symptoms, Activities of Daily Living, Quality of Life, and recreation). Several machine learning strategies were used for model development. Ten-fold cross-validation and bootstrapping were used to produce measures of overall accuracy (C-statistic, Brier Score). The sensitivity and specificity of various predicted probability cut-points were examined. RESULTS: The most accurate models produced were for the Activities of Daily Living, Pain, Symptoms, and Quality of Life subscales of the KOOS (C-statistics 0.76, 0.72, 0.72, and 0.71, respectively). Strategies varied substantially in terms of the numbers of inputs required to achieve similar accuracy, with none being superior for all outcomes. CONCLUSION: Models produced in this project provide estimates of patient-specific improvements in major outcomes 1 year after TKA. Integrating these models into clinical decision support, informed consent and shared decision making could improve patient selection, education, and satisfaction. LEVEL OF EVIDENCE: Level III, diagnostic study.

Comparing Methods to Determine the Minimal Clinically Important Differences in Patient-Reported Outcome Measures for Veterans Undergoing Elective Total Hip or Knee Arthroplasty in Veterans Health Administration Hospitals.

Importance: The minimal clinically important difference (MCID) in a patient-reported outcome measure (PROM) is the smallest change that patients perceive as beneficial. Accurate MCIDs are required when PROMs are used to evaluate the value of surgical interventions. Objective: To use well-defined distribution-based and anchor-based methods to calculate MCIDs in the Hip Disability and Osteoarthritis Outcome Score (HOOS) and in the Knee Injury and Osteoarthritis Outcome Score (KOOS) for veterans undergoing primary total hip arthroplasty or total knee arthroplasty. Design, Setting, and Participants: A prospective cohort study was conducted of 858 patients undergoing total joint replacement between March 16, 2015, and March 9, 2017, at 3 high-complexity Veterans Affairs Medical Centers. Interventions: Patients undergoing total hip arthroplasty or total knee arthroplasty were administered HOOS or KOOS PROMs prior to and 1 year after surgery. The Self-Administered Patient Satisfaction Scale (SAPS) for primary hip or knee arthroplasty was administered at 1-year follow-up as an anchor PROM. Main Outcomes and Measures: The HOOS and KOOS before and 1 year after surgery, change scores (difference between postoperative and preoperative PROM scores), and MCIDs for each measure. For anchor-based methods, receiver operating characteristic curve analysis was performed, including calculation of the area under the curve. Results: The mean (SD) age of the 271 patients who underwent hip arthroplasty was 65.6 (8.3) years, and the mean (SD) age of the 587 patients who underwent knee arthroplasty was 66.1 (8.2) years. There were 547 men in the knee arthroplasty cohort and 256 men in the hip arthroplasty cohort (total, 803 men). There were significant improvements in the mean values of every PROM, with mean (SD) differences greater than 39 for HOOS Joint Replacement (JR) and every hip subscale (HOOS JR, 39.7 [20.2]; pain, 47.6 [20.5]; symptoms, 45.1 [21.5]; activities of daily living, 43.7 [22.1]; recreation, 49.2 [33.5]; quality of life, 50.3 [27.8]) and mean (SD) differences greater than 29 for KOOS JR and every knee subscale (KOOS JR, 30.4 [17.5]; pain, 38.0 [20.4]; symptoms, 29.5 [22.1]; activities of daily living, 34.8 [20.5]; recreation, 34.6 [31.1]; quality of life, 35.2 [26.8]). Different calculation methods yielded a wide range of MCIDs. Distribution-based approaches tended to give lower values than the anchor-based approaches, which gave similar values for most PROMs. Area under the curve values demonstrated good to excellent discrimination for SAPS for nearly all PROMs. Conclusions and Relevance: Minimal clinically important difference estimates can be highly variable depending on the method used. Patient satisfaction measured by SAPS is a suitable anchor for the HOOS and KOOS. This study suggests that the SAPS-anchored MCID values presented here be used in future studies of total hip arthroplasty and total knee arthroplasty for veterans.