Use of the Fundamentals of Arthroscopic Surgical Training Workstation for Immediate Objective Feedback During Training Improves Hand-Tied Surgical Knot Proficiency: A Randomized Controlled Study.

INTRODUCTION: In a prior study, we validated the FAST workstation as an objective evaluator of hand-tied surgical knots. The goal of this study was to determine whether guided practice with the FAST workstation could lead to objective improvement in hand-tied surgical knot performance. METHODS: Sixty participants were randomized into 1 of 2 groups: a control group, with access to only the knot-tying station, and a test group, with access to the knot-tying station plus the knot testing station. The study was divided into 3 phases: prepractice, practice, and postpractice. Using the FAST workstation, participants hand-tied 5 knots, 15 knots, and 5 knots using 0 Vicryl suture in the prepractice, practice, and postpractice phases, respectively. Knots for each participant from the prepractice and postpractice phases were collected, tested, and compared. RESULTS: Within the control group, the average number of prepractice knot successes was 2.63, and the average number of postpractice knot successes was 3.40, which resulted in an improvement of average knot successes of 0.77 (P < 0.01). Within the test group, the average number of prepractice knot successes was 2.40, and the average number of postpractice knot successes was 4.10, which resulted in an improvement of average knot successes of 1.70 (P < 0.01). The difference in average improvement of knot-tying successes (0.93 knots) between the 2 groups was statistically significant (P < 0.01). DISCUSSION: A knot-tying trainer that provides objective feedback on knot performance may better improve hand-tied surgical knot proficiency compared with one without objective feedback.

Editorial Commentary: Inexpensive Training Tools Can Facilitate Basic Arthroscopy Education.

Bench-top arthroscopy training requires some form of camera visualization. Use of clinical arthroscopy equipment in a virtual reality simulation lab may be cost prohibitive for some training programs. There are creative ways to build homemade, inexpensive camera setups for basic arthroscopy skills training.

Validation of the FAST Workstation as an Objective Evaluator of Hand-Tied Surgical Knots.

INTRODUCTION: Hand-tying surgical knots is an important skill that is applicable across all surgical disciplines. Knot quality is traditionally assessed in the operating room by visual inspection, which is inherently subjective. With the increasing prevalence of simulation training, surgical trainees having access to objective feedback of their knot-tying skills are paramount for skill evaluation. Our study aimed to validate a mechanical suture loop security tester as an objective evaluation method of hand-tied surgical knot quality. METHODS: Seventy-three subjects were recruited and categorized based on surgical expertise into one of the following three groups: medical students, surgical residents, and attending surgeons. They hand-tied five surgical knots consecutively using 0 Vicryl sutures. The knots were tested using the suture loop security workstation with 10 lbs of linear force for 10 seconds. Success of the suture loop was set at less than 3 mm of loop expansion. This protocol was based on previous research showing that 3 mm of loop expansion would represent clinical failure of the knot in vivo. RESULTS: The students, residents, and attending surgeons tied a median of two, three, and four successful knots, respectively. Post hoc pairwise comparisons revealed that residents tied more successful knots than students (p < 0.001), and attending physicians tied more successful knots than students (p < 0.001). However, there was no statistically significant difference in the number of successes between residents and attending physicians (p = 0.24). CONCLUSIONS: Using the mechanical loop security workstation demonstrates construct validity as a reliable objective evaluation tool of hand-tied surgical knots.

Editorial Commentary: What's Knot to Like? In the Biomechanics Laboratory, Knotless Shoulder Anchors Hold Their Own.

Arthroscopic knot tying remains challenging, and the knots are a weak link for many repair constructs. Knotless strategies continue to evolve, and in some cases, these methods enhance biomechanical repair characteristics. The associated impact on clinical outcomes remains unknown.

Inter-rater Reliability for Metrics Scored in a Binary Fashion-Performance Assessment for an Arthroscopic Bankart Repair.

PURPOSE: To determine the inter-rater reliability (IRR) of a procedure-specific checklist scored in a binary fashion for the evaluation of surgical skill and whether it meets a minimum level of agreement (≥0.8 between 2 raters) required for high-stakes assessment. METHODS: In a prospective randomized and blinded fashion, and after detailed assessment training, 10 Arthroscopy Association of North America Master/Associate Master faculty arthroscopic surgeons (in 5 pairs) with an average of 21 years of surgical experience assessed the video-recorded 3-anchor arthroscopic Bankart repair performance of 44 postgraduate year 4 or 5 residents from 21 Accreditation Council for Graduate Medical Education orthopaedic residency training programs from across the United States. RESULTS: No paired scores of resident surgeon performance evaluated by the 5 teams of faculty assessors dropped below the 0.8 IRR level (mean = 0.93; range 0.84-0.99; standard deviation = 0.035). A comparison between the 5 assessor groups with 1 factor analysis of variance showed that there was no significant difference between the groups (P = .205). Pearson's product-moment correlation coefficient revealed a strong and statistically significant negative correlation, that is, -0.856 (P < .000), indicating that as intra-operative error rate scores increased, the IRR decreased. CONCLUSIONS: Arthroscopy Association of North America shoulder faculty raters from across the United States showed high levels of IRR in the assessment of an arthroscopic 3-anchor Bankart repair procedure. All paired assessments were above the 0.8 level and the mean IRR of all resident assessments was 0.93, indicating that they could be used for high-stakes decisions. CLINICAL RELEVANCE: With the move toward outcomes-based performance evaluation for graduate medical education, high-stakes assessments of surgical skill will require robust, reliable measurement tools that are able to withstand challenge. Surgical checklists employing metrics scored in a binary fashion meet the need and can show a high (>80%) IRR.

Editorial Commentary: Technical Skill in Arthroscopic Simulation Training: Are Wearable Motion Sensors a Step Forward?

Wearable inertial sensors can be used for assessment of surgeon body motion during arthroscopy, with demonstrated macroergonomic differences observed between novices and experts during diagnostic arthroscopy of a right cadaver knee. Before the adoption of arthroscopy training benchmarks based on inertial sensor technology, validation research should cross-correlate ergonomic variables with surgical quality.

Editorial Commentary: Fixation of Soft-Tissue Anterior Cruciate Ligament Grafts in Osteoporotic Bone.

Tibial-side fixation of soft-tissue grafts in osteoporotic bone has become an important concern, as anterior cruciate ligament reconstruction indications push toward older patients. Although soft-tissue fixation with an interference screw and cortical button may be statistically stronger than graft fixation with a screw or button alone, this hybrid construct is probably not optimized from a clinical perspective.

Suture spanning augmentation of single-row rotator cuff repair: a biomechanical analysis.

BACKGROUND: This in vitro study evaluated the biomechanical benefit of adding spanning sutures to single-row rotator cuff repair. METHODS: Mechanical testing was performed to evaluate 9 pairs of cadaveric shoulders with complete rotator cuff repairs, with a single-row technique used on one side and the suture spanning technique on the other. The spanning technique included sutures from 2 lateral anchors securing tendon near the musculotendinous junction, spanning the same anchor placement from single-row repair. The supraspinatus muscle was loaded to 100 N at 0.25 Hz for 100 cycles, followed by a ramp to failure. Markers and a video tracking system measured anterior and posterior gap formation across the repair at 25-cycle intervals. The force at which the stiffness decreased by 50% and 75% was determined. Data were compared using paired t-tests. RESULTS: One single-row repair failed at <25 cycles. Both anterior and posterior gap distances tended to be 1 to 2 mm larger for the single-row repairs than for the suture spanning technique. The difference was statistically significant at all cycles for the posterior gap formation (P ≤ .02). The trends were not significant for the anterior gap (P ≥ .13). The loads at which the stiffness decreased by 50% and 75% did not differ significantly between the 2 types of repair (P ≥ .10). CONCLUSIONS: The suture spanning technique primarily improved posterior gap formation. Decreased posterior gap formation could reduce failure rates for rotator cuff repair.

Editorial Commentary: Improving Proficiency in Arthroscopic Knot Tying: Focus on Performance Over Appearance.

Arthroscopic knot tying is a challenging motor skill that is best learned at the benchtop, before surgery on patients. Recent studies suggest that these skills can be improved by direct assessment of knot biomechanics, providing the learner with immediate and objective feedback. To optimize proficiency at arthroscopic knot tying, we focus on performance over appearance.

Retention of Skills After Simulation-based Training in Orthopaedic Surgery.

Simulation-based surgical skills training has become essential in orthopaedic practice because of concerns about patient safety and an increase in technically challenging procedures. Surgical skills training in specifically designed simulation laboratories allows practice of procedures in a risk-free environment before they are performed in the operating room. The transferability of acquired skills to performance with patients is the most effective measure of the predictive validity of simulation-based training. Retention of the skills transferred to clinical situations is also critical. However, evidence of simulation-based skill retention in the orthopaedic literature is limited, and concerns about sustainability exist. Solutions for skill decay include repeated practice of the tasks learned on simulators and reinforcement of areas that are sensitive to decline. Further research is required to determine the retention rates of surgical skills acquired in simulation-based training as well as the success of proposed solutions for skill decay.

Simulation-based summative assessments in surgery.

BACKGROUND: The American College of Surgeons-Accredited Education Institutes (ACS-AEI) Consortium aims to enhance patient safety and advance surgical education through the use of cutting-edge simulation-based training and assessment methods. The annual ACS-AEI Consortium meeting provides a forum to discuss the latest simulation-based training and assessment methods and includes special panel presentations on key topics. METHODS: During the 8th annual Consortium, there was a panel presentation on simulation-based summative assessments, during which experiences from across surgical disciplines were presented. The formal presentations were followed by a robust discussion between the conference attendees and the panelists. RESULTS: This report summarizes the panelists' presentations and their ensuing discussion with attendees. CONCLUSION: The focus of this report is on the basis for and advances in simulation-based summative assessments, the current practices employed across various surgical disciplines, and future directions that may be pursued by the ACS-AEI Consortium.

Asymmetry in Dominant / Non-Dominant Hand Performance Differentiates Novices from Experts on an Arthroscopy Virtual Reality Serious Game.

Safe and effective arthroscopic surgery requires ambidextrous motor skills. The current study examined dominant versus non-dominant hand performance on a virtual reality serious game in a group of expert arthroscopic surgeons (n=15) compared to a group of orthopedic surgery residents (n=10). A virtual reality Tetris game was performed with the arthroscopic camera and arthroscope in one hand, using an arthroscopic grasping tool in the opposite hand to manipulate the virtual Tetris blocks onto the game grid. A second run was performed after swapping instruments between hands. The order of hand testing was randomized. There was no statistically significant difference in exercise time, grasper path length, or camera path length between the right and left hands of the expert surgeons. In contrast, there were statistically significant differences in all of these parameters between the two hands for the orthopedic surgery residents, with better performance when the grasping tool was used in the dominant hand. The findings of this study suggest that virtual reality games which incorporate progressive cognitive loading could be used to facilitate training, automation, and objective assessment of surgical motor skills.

A Proficiency-Based Progression Training Curriculum Coupled With a Model Simulator Results in the Acquisition of a Superior Arthroscopic Bankart Skill Set.

PURPOSE: To determine the effectiveness of proficiency-based progression (PBP) training using simulation both compared with the same training without proficiency requirements and compared with a traditional resident course for learning to perform an arthroscopic Bankart repair (ABR). METHODS: In a prospective, randomized, blinded study, 44 postgraduate year 4 or 5 orthopaedic residents from 21 Accreditation Council for Graduate Medical Education-approved US orthopaedic residency programs were randomly assigned to 1 of 3 skills training protocols for learning to perform an ABR: group A, traditional (routine Arthroscopy Association of North America Resident Course) (control, n = 14); group B, simulator (modified curriculum adding a shoulder model simulator) (n = 14); or group C, PBP (PBP plus the simulator) (n = 16). At the completion of training, all subjects performed a 3 suture anchor ABR on a cadaveric shoulder, which was videotaped and scored in blinded fashion with the use of previously validated metrics. RESULTS: The PBP-trained group (group C) made 56% fewer objectively assessed errors than the traditionally trained group (group A) (P = .011) and 41% fewer than group B (P = .049) (both comparisons were statistically significant). The proficiency benchmark was achieved on the final repair by 68.7% of participants in group C compared with 36.7% in group B and 28.6% in group A. When compared with group A, group B participants were 1.4 times, group C participants were 5.5 times, and group C(PBP) participants (who met all intermediate proficiency benchmarks) were 7.5 times as likely to achieve the final proficiency benchmark. CONCLUSIONS: A PBP training curriculum and protocol coupled with the use of a shoulder model simulator and previously validated metrics produces a superior arthroscopic Bankart skill set when compared with traditional and simulator-enhanced training methods. CLINICAL RELEVANCE: Surgical training combining PBP and a simulator is efficient and effective. Patient safety could be improved if surgical trainees participated in PBP training using a simulator before treating surgical patients.

The Bankart Performance Metrics Combined With a Cadaveric Shoulder Create a Precise and Accurate Assessment Tool for Measuring Surgeon Skill.

PURPOSE: To determine if previously validated performance metrics for an arthroscopic Bankart repair (ABR) coupled with a cadaveric shoulder are a valid assessment tool with the ability to discriminate between the performances of experienced and novice surgeons and to establish a proficiency benchmark for an ABR using a cadaveric shoulder. METHODS: Ten master/associate master faculty from an Arthroscopy Association of North America Resident Course (experienced group) were compared with 12 postgraduate year 4 and postgraduate year 5 orthopaedic residents (novice group). Each group was instructed to perform a diagnostic arthroscopy and a 3 suture anchor Bankart repair on a cadaveric shoulder. The procedure was videotaped in its entirety and independently scored in blinded fashion by a pair of trained reviewers. Scoring was based on defined and previously validated metrics for an ABR and included steps, errors, "sentinel" (more serious) errors, and time. RESULTS: The inter-rater reliability was 0.92. Novice surgeons made 50% more errors (5.86 v 2.95, P = .013), showed more performance variability (SD, 1.86 v 0.55), and took longer to perform the procedure (45.5 minutes v 25.9 minutes, P < .001). The greatest difference in errors related to suture delivery and management (exclusive of knot tying) (1.95 v 0.45, P = .024). CONCLUSIONS: The assessment tool composed of validated arthroscopic Bankart metrics coupled with a cadaveric shoulder accurately distinguishes the performance of experienced from novice orthopaedic surgeons. A benchmark based on the mean performance of the experienced group includes completion of a 3-anchor Bankart repair, and enacting no more than 3 total errors and 1 sentinel error. CLINICAL RELEVANCE: Validated procedural metrics combined with the use of a cadaveric shoulder can be used to assess the performance of an ABR. The methodology used may serve as a template for outcomes-based procedural skills training in general.

Objective Assessment of Knot-Tying Proficiency With the Fundamentals of Arthroscopic Surgery Training Program Workstation and Knot Tester.

PURPOSE: To assess a new method for biomechanical assessment of arthroscopic knots and to establish proficiency benchmarks using the Fundamentals of Arthroscopic Surgery Training (FAST) Program workstation and knot tester. METHODS: The first study group included 20 faculty at an Arthroscopy Association of North America resident arthroscopy course (19.9 ± 8.25 years in practice). The second group comprised 30 experienced surgeons attending an Arthroscopy Association of North America fall course (17.1 ± 19.3 years in practice). The training group included 44 postgraduate year 4 or 5 orthopaedic residents in a randomized, prospective study of proficiency-based training, with 3 subgroups: group A, standard training (n = 14); group B, workstation practice (n = 14); and group C, proficiency-based progression using the knot tester (n = 16). Each subject tied 5 arthroscopic knots backed up by 3 reversed hitches on alternating posts. Knots were tied under video control around a metal mandrel through a cannula within an opaque dome (FAST workstation). Each suture loop was stressed statically at 15 lb for 15 seconds. A calibrated sizer measured loop expansion. Knot failure was defined as 3 mm of loop expansion or greater. RESULTS: In the faculty group, 24% of knots "failed" under load. Performance was inconsistent: 12 faculty had all knots pass, whereas 2 had all knots fail. In the second group of practicing surgeons, 21% of the knots failed under load. Overall, 56 of 250 knots (22%) tied by experienced surgeons failed. For the postgraduate year 4 or 5 residents, the aggregate knot failure rate was 26% for the 220 knots tied. Group C residents had an 11% knot failure rate (half the overall faculty rate, P = .013). CONCLUSIONS: The FAST workstation and knot tester offer a simple and reproducible educational approach for enhancement of arthroscopic knot-tying skills. Our data suggest that there is significant room for improvement in the quality and consistency of these important arthroscopic skills, even for experienced arthroscopic surgeons. LEVEL OF EVIDENCE: Level II, prospective comparative study.

Metric Development for an Arthroscopic Bankart Procedure: Assessment of Face and Content Validity.

PURPOSE: To establish the metrics (operational definitions) necessary to characterize a reference arthroscopic Bankart procedure, and to seek consensus from experienced shoulder arthroscopists on the appropriateness of the steps, as well as errors identified. METHODS: Three experienced arthroscopic shoulder surgeons and an experimental psychologist (comprising the Metrics Group) deconstructed an arthroscopic Bankart procedure. Fourteen full-length videos were analyzed to identify the essential steps and potential errors. Sentinel (i.e., more serious) errors were defined as either (1) potentially jeopardizing the procedure outcome or (2) creating iatrogenic damage to the shoulder. The metrics were stress tested for clarity and the ability to be scored in binary fashion during a video review as either occurring or not occurring. The metrics were subjected to analysis by a panel of 27 experienced arthroscopic shoulder surgeons to obtain face and content validity using a modified Delphi Panel methodology (consensus opinion of experienced surgeons rendered by cyclical deliberations). RESULTS: Forty-five steps and 13 phases characterizing an arthroscopic Bankart procedure were identified. Seventy-seven procedural errors were specified, with 20 designated as sentinel errors. The modified Delphi Panel deliberation created the following changes: 2 metrics were deleted, 1 was added, and 5 were modified. Consensus on the resulting Bankart metrics was obtained and face and content validity verified. CONCLUSIONS: This study confirms that a core group of experienced arthroscopic surgeons is able to perform task deconstruction of an arthroscopic Bankart repair and create unambiguous step and error definitions (metrics) that accurately characterize the essential components of the procedure. Analysis and revision by a larger panel of experienced arthroscopists were able to validate the Bankart metrics. CLINICAL RELEVANCE: The ability to perform task deconstruction and validate the resulting metrics will play a key role in improving surgical skills training and assessing trainee progression toward proficiency.

The Bankart Performance Metrics Combined With a Shoulder Model Simulator Create a Precise and Accurate Training Tool for Measuring Surgeon Skill.

PURPOSE: To determine if a dry shoulder model simulator coupled with previously validated performance metrics for an arthroscopic Bankart repair (ABR) would be a valid tool with the ability to discriminate between the performance of experienced and novice surgeons, and to establish a proficiency benchmark for an ABR using a model simulator. METHODS: We compared an experienced group of arthroscopic shoulder surgeons (Arthroscopy Association of North America faculty) (n = 12) with a novice group (n = 7) (postgraduate year 4 or 5 orthopaedic residents). All surgeons were instructed to perform a diagnostic arthroscopy and a 3 suture anchor Bankart repair on a dry shoulder model. Each procedure was videotaped in its entirety and scored in blinded fashion independently by 2 trained reviewers. Scoring used previously validated metrics for an ABR and included steps, errors, and "sentinel" (more serious) errors. RESULTS: The inter-rater reliability among pairs of raters averaged 0.93. The experienced group made 63% fewer errors, committed 79% fewer sentinel errors, and performed the procedure in 42% less time than the novice group (all significant differences). The greatest difference in errors between the groups involved anchor preparation and insertion, suture delivery and management, and knot tying. CONCLUSIONS: The tool comprised by validated ABR metrics coupled with a dry shoulder model simulator is able to accurately distinguish between the performance of experienced and novice orthopaedic surgeons. A performance benchmark based on the mean performance of the experienced group includes completion of a 3 anchor Bankart repair, enacting no more than 4 total errors and 1 sentinel error. CLINICAL RELEVANCE: The combination of performance metrics and an arthroscopic shoulder model simulator can be used to improve the effectiveness of surgical skills training for an ABR. The methodology used may serve as a template for outcomes-based procedural skills training in general.