Assessment of Technical Competence in Distal Radius Fracture Fixation by a Volar Locking Plate: A Global Delphi Consensus Study.

PURPOSE: Volar locking plate fixation of distal radius fractures is a common orthopedic procedure and should be mastered by graduating orthopedic residents. Surgical education is transitioning from a traditional time-based approach to competency-based medical education. Valid and objective assessment is essential for successful transition. The purpose of this study was to develop a comprehensive, procedure-specific assessment tool to evaluate technical competence in volar locking plate osteosynthesis of a distal radius fracture. METHODS: International orthopedic/trauma experts involved in resident education participated as panelists in a four-round online Delphi process to reach consensus on the content of the assessment tool. Round 1 was an item-generating round, in which the panelists identified potential assessment parameters. In round 2, the panelists rated the importance of each suggested assessment parameter and reached consensus on which to include in the assessment tool. Round 3 yielded specific assessment score intervals for specific bone and fracture models and is not reported in this study. In round 4, the panelists assigned weights to the assessment parameters on a 1-10 scale to determine how each parameter should have an impact on the overall results. RESULTS: Eighty-seven surgeons, representing 42 countries, participated in the study. Round 1 resulted in 45 assessment parameters, grouped into five procedural steps. After round 2, the number of parameters was reduced to 39. After the final round, an additional parameter was removed and weights were assigned to the remaining parameters. CONCLUSIONS: Using a systematic methodology, a preliminary assessment tool to evaluate technical competence in distal radius fracture fixation was developed. A consensus of international experts supports the content validity of the assessment tool. CLINICAL RELEVANCE: This assessment tool represents the first step in the evidence-based assessment essential for competency-based medical education. Before implementation, further studies exploring validity of variations of the assessment tool in different educational contexts are required.

Practicing Procedural Skills Is More Effective Than Basic Psychomotor Training in Knee Arthroscopy: A Randomized Study.

BACKGROUND: Simulator-assisted arthroscopy education traditionally consists of initial training of basic psychomotor skills before advancing to more complex procedural tasks. PURPOSE: To explore and compare the effects of basic psychomotor skills training versus procedural skills training on novice surgeons' subsequent simulated knee arthroscopy performance. STUDY DESIGN: Controlled laboratory study. METHODS: Overall, 22 novice orthopaedic surgeons and 11 experienced arthroscopic surgeons participated in this study, conducted from September 2015 to January 2017. Novices received a standardized introductory lesson on knee arthroscopy before being randomized into a basic skills training group or a procedural skills training group. Each group performed 2 sessions on a computer-assisted knee arthroscopy simulator: The basic skills training group completed 1 session consisting of basic psychomotor skills modules and 1 session of procedural modules (diagnostic knee arthroscopy and meniscal resection), whereas the procedural skills training group completed 2 sessions of procedural modules. Performance of the novices was compared with that of the experienced surgeons to explore evidence of validity for the basic psychomotor training skills modules and the procedural modules. The effect of prior basic psychomotor skills training and procedural skills training was explored by comparing pre- and posttraining performances of the randomized groups using a mixed-effects regression model. RESULTS: Validity evidence was found for the procedural modules, as test results were reliable and experienced surgeons significantly outperformed novices. We found no evidence of validity for the basic psychomotor skills modules, as test scores were unreliable and there was no difference in performance between the experienced surgeons and novices. We found no statistical effect of basic psychomotor skills training as compared with no training (P = .49). We found a statistically significant effect of prior procedural skills training (P < .001) and a significantly larger effect of procedural skills training as compared with basic psychomotor skills training (P = .019). CONCLUSION: Procedural skills training was significantly more effective than basic psychomotor skills training regarding improved performance in diagnostic knee arthroscopy and meniscal resection on a knee arthroscopy simulator. Furthermore, the basic psychomotor skills modules lacked validity evidence. CLINICAL RELEVANCE: On the basis of these results, we suggest that future competency-based curricula focus their training on full knee arthroscopy procedures. This could improve future education programs.

Testing basic competency in knee arthroscopy using a virtual reality simulator: exploring validity and reliability.

BACKGROUND: Diagnostic knee arthroscopy is a common procedure that orthopaedic residents are expected to learn early in their training. Arthroscopy requires a different skill set from traditional open surgery, and many orthopaedic residents feel less prepared for arthroscopic procedures. Virtual reality simulation training and testing provide an opportunity to ensure basic competency before proceeding to supervised procedures in patients. METHODS: Twenty-six physicians (thirteen novices and thirteen experienced arthroscopic surgeons) were voluntarily recruited to perform a test consisting of five arthroscopic procedures on a knee arthroscopy simulator. Performance was evaluated by obtaining predefined metrics from the simulator for each procedure, and z-scores, describing suboptimal performance, were calculated from the metrics. The intercase reliability of the simulator metrics was explored by calculating an intraclass correlation coefficient. Finally, a pass-or-fail standard was set with use of the contrasting groups method, and the consequences of the pass-or-fail standard were explored. RESULTS: One procedure was excluded from the final test because of a lack of validity. The total Z-scores for the four procedures included in the final test showed an intercase reliability of 0.87 (95% confidence interval, 0.78 to 0.93). The total mean z-score (and standard deviation) was 38.6 ± 27.3 points for the novices and 0.0 ± 9.1 points for the experienced surgeons (p < 0.0005). The pass-or-fail standard was set at a total z-score of 15.5 points, resulting in two of the novices passing the test and a single experienced surgeon failing the test. CONCLUSIONS: By combining four procedures on a virtual reality arthroscopy simulator, it was possible to create a valid, reliable, and feasible test of basic arthroscopic competency and to establish a credible pass-or-fail standard. CLINICAL RELEVANCE: The simulation-based test and pass-or-fail standard could aid in assessing and ensuring basic competency of future orthopaedic residents before proceeding to supervised procedures in patients.