Efficiency and Accuracy of Computerized Adaptive Testing for the Oswestry Disability Index and Neck Disability Index.

This study aimed to determine the efficiency and accuracy of computerized adaptive testing (CAT) models of the Oswestry Disability Index (ODI) and Neck Disability Index (NDI). Methods: The study involved simulation using retrospectively collected real-world data. Previously developed CAT models of the ODI and NDI were applied to the responses from 52,551 and 18,196 patients with spinal conditions, respectively. Efficiency was evaluated by the reduction in the number of questions administered. Accuracy was evaluated by comparing means and standard deviations, calculating Pearson r and intraclass correlation coefficient (ICC) values, plotting the frequency distributions of CAT and full questionnaire scores, plotting the frequency distributions of differences between paired scores, and Bland-Altman plotting. Score changes, calculated as the postoperative ODI or NDI scores minus the preoperative scores, were compared between the CAT and full versions in patients for whom both preoperative and postoperative ODI or NDI questionnaires were available. Results: CAT models of the ODI and NDI required an average of 4.47 and 4.03 fewer questions per patient, respectively. The mean CAT ODI score was 0.7 point lower than the full ODI score (35.4 ± 19.0 versus 36.1 ± 19.3), and the mean CAT NDI score was 1.0 point lower than the full NDI score (34.7 ± 19.3 versus 33.8 ± 18.5). The Pearson r was 0.97 for both the ODI and NDI, and the ICC was 0.97 for both. The frequency distributions of the CAT and full scores showed marked overlap for the ODI and NDI. Differences between paired scores were less than the minimum clinically important difference in 98.9% of cases for the ODI and 98.5% for the NDI. Bland-Altman plots showed no proportional bias. The ODI and NDI score changes could be calculated in a subgroup of 6,044 and 4,775 patients, respectively; the distributions of the ODI and NDI score changes were near identical between the CAT and full versions. Conclusions: CAT models were able to reduce the question burden of the ODI and NDI. Scores obtained from the CAT models were faithful to those from the full questionnaires, both on the population level and on the individual patient level. Level of Evidence: Prognostic Level III. See Instructions for Authors for a complete description of levels of evidence.

Computerized adaptive testing for the Oxford Hip, Knee, Shoulder, and Elbow scores : accurate measurement from fewer, and more patient-focused, questions.

AIMS: The aim of this study was to develop and evaluate machine-learning-based computerized adaptive tests (CATs) for the Oxford Hip Score (OHS), Oxford Knee Score (OKS), Oxford Shoulder Score (OSS), and the Oxford Elbow Score (OES) and its subscales. METHODS: We developed CAT algorithms for the OHS, OKS, OSS, overall OES, and each of the OES subscales, using responses to the full-length questionnaires and a machine-learning technique called regression tree learning. The algorithms were evaluated through a series of simulation studies, in which they aimed to predict respondents' full-length questionnaire scores from only a selection of their item responses. In each case, the total number of items used by the CAT algorithm was recorded and CAT scores were compared to full-length questionnaire scores by mean, SD, score distribution plots, Pearson's correlation coefficient, intraclass correlation (ICC), and the Bland-Altman method. Differences between CAT scores and full-length questionnaire scores were contextualized through comparison to the instruments' minimal clinically important difference (MCID). RESULTS: The CAT algorithms accurately estimated 12-item questionnaire scores from between four and nine items. Scores followed a very similar distribution between CAT and full-length assessments, with the mean score difference ranging from 0.03 to 0.26 out of 48 points. Pearson's correlation coefficient and ICC were 0.98 for each 12-item scale and 0.95 or higher for the OES subscales. In over 95% of cases, a patient's CAT score was within five points of the full-length questionnaire score for each 12-item questionnaire. CONCLUSION: Oxford Hip Score, Oxford Knee Score, Oxford Shoulder Score, and Oxford Elbow Score (including separate subscale scores) CATs all markedly reduce the burden of items to be completed without sacrificing score accuracy.Cite this article: Bone Jt Open 2022;3(10):786-794.

Machine Learning Can Predict Level of Improvement in Shoulder Arthroplasty.

The ability to accurately predict postoperative outcomes is of considerable interest in the field of orthopaedic surgery. Machine learning has been used as a form of predictive modeling in multiple health-care settings. The purpose of the current study was to determine whether machine learning algorithms using preoperative data can predict improvement in American Shoulder and Elbow Surgeons (ASES) scores for patients with glenohumeral osteoarthritis (OA) at a minimum of 2 years after shoulder arthroplasty. METHODS: This was a retrospective cohort study that included 472 patients (472 shoulders) diagnosed with primary glenohumeral OA (mean age, 68 years; 56% male) treated with shoulder arthroplasty (431 anatomic total shoulder arthroplasty and 41 reverse total shoulder arthroplasty). Preoperative computed tomography (CT) scans were used to classify patients on the basis of glenoid and rotator cuff morphology. Preoperative and final postoperative ASES scores were used to assess the level of improvement. Patients were separated into 3 improvement ranges of approximately equal size. Machine learning methods that related patterns of these variables to outcome ranges were employed. Three modeling approaches were compared: a model with the use of all baseline variables (Model 1), a model omitting morphological variables (Model 2), and a model omitting ASES variables (Model 3). RESULTS: Improvement ranges of ≤28 points (class A), 29 to 55 points (class B), and >55 points (class C) were established. Using all follow-up time intervals, Model 1 gave the most accurate predictions, with probability values of 0.94, 0.95, and 0.94 for classes A, B, and C, respectively. This was followed by Model 2 (0.93, 0.80, and 0.73) and Model 3 (0.77, 0.72, and 0.71). CONCLUSIONS: Machine learning can accurately predict the level of improvement after shoulder arthroplasty for glenohumeral OA. This may allow physicians to improve patient satisfaction by better managing expectations. These predictions were most accurate when latent variables were combined with morphological variables, suggesting that both patients' perceptions and structural pathology are critical to optimizing outcomes in shoulder arthroplasty. LEVEL OF EVIDENCE: Therapeutic Level IV. See Instructions for Authors for a complete description of levels of evidence.

The Application of Computerized Adaptive Testing to the International Knee Documentation Committee Subjective Knee Evaluation Form.

BACKGROUND: Patient-reported outcome measures (PROMs) are commonly used to monitor functional outcomes for clinical and research purposes; unfortunately, many PROMs include redundant, burdensome questions for patients. The use of predictive models to implement computerized adaptive testing (CAT) offer a potential solution to reduce question burden in outcomes research. PURPOSE: To validate the usage of an appropriate CAT system to improve the efficiency of the International Knee Documentation Committee (IKDC) Subjective Knee Form. STUDY DESIGN: Cohort study (Diagnosis); Level of evidence, 2. METHODS: Validation was based on electronically collected patient responses from 2 separate orthopaedic sports medicine clinics. Diagnoses included, but were not limited to, meniscal lesions, ligamentous injuries, and chondral defects. The CAT system was previously developed through analysis of an electronic knee PROM database that did not contain any of these cases. RESULTS: A total of 2173 patient responses (1229 patients) were collected. The CAT model was able to reduce the question burden by a mean of 9.33 questions (45.1%). Higher CAT-predicted scores correlated strongly with higher actual scores (r = 0.99; intraclass correlation coefficient = 0.99). The mean difference between the CAT-predicted score and the actual PROM score was 0.48 of a point on a scale of 0 to 100. CONCLUSION: The use of CAT systems, in conjunction with electronic PROMs, can accurately predict outcome scores for IKDC PROMs, while dramatically decreasing the number of questionnaire items needed for any given patient. By decreasing questionnaire burden, clinicians and researchers can potentially increase patient participation and follow-up in both clinical assessments and research trials.

Improving Efficiency of Patient-Reported Outcome Collection: Application of Computerized Adaptive Testing to DASH and QuickDASH Outcome Scores.

PURPOSE: Patient-reported outcome measures assess health status and treatment outcomes in orthopedic care, but they may burden patients with lengthy questionnaires. Predictive models using machine learning, known as computerized adaptive testing (CAT), offer a potential solution. This study evaluates the ability of CAT to improve efficiency of the 30-item Disabilities of the Arm, Shoulder, and Hand (DASH) and 11-item QuickDASH questionnaires. METHODS: A total of 2,860 DASH and 27,355 QuickDASH respondents were included in the analysis. The CAT system was retrospectively applied to each set of patient responses stored on the instrument to calculate a CAT-specific score for all DASH and QuickDASH entries. The accuracy of the CAT scores, viewed in the context of the minimal clinically important difference for both patient-reported outcome measures (DASH, 12; QuickDASH, 9), was determined through descriptive statistics, Pearson correlation coefficient, intraclass correlation coefficient, and distribution of scores and score differences. RESULTS: The CAT model required an average of 15.3 questions to be answered for the DASH and 5.8 questions for the QuickDASH, representing a 49% and 47% decrease in question burden, respectively. Mean CAT score was the same for DASH and 0.1 points lower for QuickDASH with similar SDs (DASH, 12.9 ± 19.8 vs 12.9 ± 19.9; QuickDASH, 32.7 ± 24.7 vs 32.6 ± 24.6). Pearson coefficients (DASH, 0.99; QuickDASH, 0.98) and intraclass correlation coefficients (DASH, 1.0; QuickDASH, 0.98) indicated strong agreement between scores. The difference between the CAT and full score was less than the minimal clinically important difference in 99% of cases for DASH and approximately 95% of cases for QuickDASH. CONCLUSIONS: The application of CAT to DASH and QuickDASH surveys demonstrated an ability to lessen the response burden with negligible effect on score integrity. CLINICAL RELEVANCE: In the case of DASH and QuickDASH, CAT is an appropriate alternative to full questionnaire implementation for patient outcome score collection.

Application of Computerized Adaptive Testing to the Foot and Ankle Ability Measure.

BACKGROUND: Patient-reported outcome measures are an increasingly important tool for assessing the impact of treatments orthopedic surgeons render. Despite their importance, they can present a burden. We examined the validity and utility of a computerized adaptive testing (CAT) method to reduce the number of questions on the Foot and Ankle Ability Measure (FAAM), a validated anatomy-specific outcome measure. METHODS: A previously developed FAAM CAT system was applied to the responses of patients undergoing foot and ankle evaluation and treatment over a 3-year period (2017-2019). A total of 15 902 responses for the Activities of Daily Living (ADL) subscale and a total of 14 344 responses for the Sports subscale were analyzed. The accuracy of the CAT to replicate the full-form score was assessed. RESULTS: The CAT system required 11 questions to be answered for the ADL subscale in 85.1% of cases (range, 11-12). The number of questions answered on the Sports subscale was 6 (range, 5-6) in 66.4% of cases. The mean difference between the full FAAM ADL subscale and CAT was 0.63 of a point. The mean difference between the FAAM Sports subscale and CAT was 0.65 of a point. CONCLUSION: The FAAM CAT was able to reduce the number of responses a patient would need to answer by nearly 50%, while still providing a valid outcome score. This measure can therefore be directly correlated with previously obtained full FAAM scores in addition to providing a foot/ankle-specific measure, which previously reported CAT systems are not able to do. LEVEL OF EVIDENCE: Level IV, case series.

Use of Computerized Adaptive Testing to Develop More Concise Patient-Reported Outcome Measures.

BACKGROUND: Patient-reported outcome measures (PROMs) are essential tools that are used to assess health status and treatment outcomes in orthopaedic care. Use of PROMs can burden patients with lengthy and cumbersome questionnaires. Predictive models using machine learning known as computerized adaptive testing (CAT) offer a potential solution. The purpose of this study was to evaluate the ability of CAT to improve efficiency of the Veterans RAND 12 Item Health Survey (VR-12) by decreasing the question burden while maintaining the accuracy of the outcome score. METHODS: A previously developed CAT model was applied to the responses of 19,523 patients who had completed a full VR-12 survey while presenting to 1 of 5 subspecialty orthopaedic clinics. This resulted in the calculation of both a full-survey and CAT-model physical component summary score (PCS) and mental component summary score (MCS). Several analyses compared the accuracy of the CAT model scores with that of the full scores by comparing the means and standard deviations, calculating a Pearson correlation coefficient and intraclass correlation coefficient, plotting the frequency distributions of the 2 score sets and the score differences, and performing a Bland-Altman assessment of scoring patterns. RESULTS: The CAT model required 4 fewer questions to be answered by each subject (33% decrease in question burden). The mean PCS was 1.3 points lower in the CAT model than with the full VR-12 (41.5 ± 11.0 versus 42.8 ± 10.4), and the mean MCS was 0.3 point higher (57.3 ± 9.4 versus 57.0 ± 9.6). The Pearson correlation coefficients were 0.97 for PCS and 0.98 for MCS, and the intraclass correlation coefficients were 0.96 and 0.97, respectively. The frequency distribution of the CAT and full scores showed significant overlap for both the PCS and the MCS. The difference between the CAT and full scores was less than the minimum clinically important difference (MCID) in >95% of cases for the PCS and MCS. CONCLUSIONS: The application of CAT to the VR-12 survey demonstrated an ability to lessen the response burden for patients with a negligible effect on score integrity.

A concise shoulder outcome measure: application of computerized adaptive testing to the American Shoulder and Elbow Surgeons Shoulder Assessment.

BACKGROUND: Patient-reported outcome measures enable quantitative and patient-centric assessment of orthopedic interventions; however, increased use of these forms has an associated burden for patients and practices. We examined the utility of a computerized adaptive testing (CAT) method to reduce the number of questions on the American Shoulder and Elbow Surgeons (ASES) instrument. METHODS: A previously developed ASES CAT system was applied to the responses of 2763 patients who underwent shoulder evaluation and treatment and had answered all questions on the full ASES instrument. Analyses to assess the accuracy of the CAT score in replicating the full-form score included the mean and standard deviation of both groups of scores, frequency distributions of the 2 sets of scores and score differences, Pearson and intraclass correlation coefficients, and Bland-Altman assessment of patterns in score differences. RESULTS: By tailoring questions according to prior responses, CAT reduced the question burden by 40%. The mean difference between CAT and full ASES scores was -0.14, and the scores were within 5 points in 95% of cases (a 12-point difference is considered the threshold for clinical significance) and were clustered around zero. The correlation coefficients were 0.99, and the frequency distributions of the CAT and full ASES scores were nearly identical. The differences between scores were independent of the overall score, and no significant bias for CAT scores was found in either a positive or negative direction. CONCLUSION: The ASES CAT system lessens respondent burden with a negligible effect on score integrity.