Live Observational Objective Assessment of Operative Performance in a Cadaveric Model is Equivalent to Delayed Video-Based Assessment.

PURPOSE: The purpose of our study was to compare real-time, live observational scoring with delayed retrospective video review of operative performance and to determine whether the evaluation method affected the attainment of proficiency benchmarks. METHODS: Sixteen arthroscopy/sports medicine fellows and 2 senior residents completed training to perform arthroscopic Bankart repairs (ABRs) and arthroscopic rotator cuff repairs (ARCRs) using a proficiency-based progression curriculum. Each final operative performance for 15 randomly selected ABRs and 13 ARCRs performed on cadavers were scored live (observation during the operative performance) and on delayed video review (6-8 weeks) by 1 of 15 trained raters using validated metric-based (step and error) assessment tools. The inter-rater reliability (IRR) of live versus video review by a single rater was calculated, and changes to the trainee's attainment of the proficiency benchmarks were noted. The correlation coefficient (r) and the R2 were also calculated for the paired scores from the randomly selected performances. RESULTS: No significant differences in the observed IRR agreement or the attainment of the proficiency benchmarks were found when comparing live to video assessment for either ABR or ARCR. The correlation coefficients r and R2 were considerably lower than the agreement coefficient (IRR) for rotator cuff steps (e.g., R2 = 0.74 vs. IRR = 0.97, P = 0.001); Bankart errors (R2 = 0.73 vs. IRR = 0.98, P = 0.006); and rotator cuff errors (R2 = 0.48 vs. IRR = 0.98, P = 0.0002). CONCLUSIONS: Real-time live and delayed video-based scoring of operative performance are essentially equivalent for the metric-based assessments of operative performance in ABRs and ARCRs. When the IRR agreement coefficient was compared with the correlation coefficients, the former was found to have greater homogeneity and measurement precision. CLINICAL RELEVANCE: Metric-based live scoring is reliable and accurate for operative performance assessment, including high-stakes evaluations.

Isolated bioinductive repair of partial-thickness rotator cuff tears using a resorbable bovine collagen implant: two-year radiologic and clinical outcomes from a prospective multicenter study.

BACKGROUND: Current surgical treatment options for partial-thickness tears (eg, takedown and repair, in situ repair) are limited by the degenerative nature of the underlying tendon and may require extensive intervention that can alter the anatomic footprint. The complexity of available techniques to address these issues led to the development of a resorbable collagen implant, which can be used to create a bioinductive repair of partial-thickness tears. METHODS: We prospectively enrolled 33 patients with chronic, degenerative, intermediate-grade (n = 12), or high-grade (n = 21) partial-thickness tears (11 articular, 10 bursal, 4 intrasubstance, and 8 hybrid) of the supraspinatus tendon in a multicenter study. After arthroscopic subacromial decompression without a traditional rotator cuff repair, a bioinductive implant was secured over the bursal surface of the tendon. Clinical outcomes were assessed using American Shoulder and Elbow Surgeons (ASES) and Constant-Murley scores (CMS) preoperatively and at 3 months, 1 year, and 2 years postoperatively. Magnetic resonance imaging was performed to assess postoperative tendon healing and thickness at the original tear site. RESULTS: At 2-year follow-up, mean ASES and CMS scores improved both clinically and statistically at 1 and 2 years, compared with baseline, for intermediate- and high-grade tears. There was magnetic resonance imaging evidence of new tissue fill-in within the original baseline tear in 100% of the intermediate-grade tears and 95% of the high-grade tears. In 90.9% of the intermediate-grade tears and 84.2% of the high-grade tears, this new tissue fill-in represented at least an additional 50% of the volume of the initial lesion. From baseline to 2-year follow-up, the mean tendon thickness increased by 1.2 mm (standard deviation, 1.3; P = .012) and 1.8 mm (standard deviation, 2.2; P = .003) in the intermediate- and high-grade tears, respectively. The analysis of tear grade and location revealed no statistically significant difference in the change in mean tendon thickness at any time point. One patient with a high-grade articular lesion demonstrated progression to a full-thickness tear; however, the patient was noncompliant and the injury occurred while shoveling snow 1 month after surgery. Neither tear location nor treatment of bicep pathology affected the ASES or CMS scores at any follow-up point. No serious adverse events related to the implant were reported. CONCLUSION: Final results from this 2-year prospective study indicate that the use of this resorbable bovine collagen implant for isolated bioinductive repair of intermediate- and high-grade partial-thickness rotator cuff tears of the supraspinatus is safe and effective, regardless of tear grade and location.

A Proficiency-Based Progression Simulation Training Curriculum to Acquire the Skills Needed in Performing Arthroscopic Bankart and Rotator Cuff Repairs-Implementation and Impact.

PURPOSE: To investigate the impact of a proficiency-based progression (PBP) curriculum employed to teach trainees in the skills needed to demonstrate proficiency for an arthroscopic Bankart repair (ABR) and an arthroscopic rotator cuff repair (ARCR) by objectively comparing pre- and immediate postcourse performances. METHODS: In a prospective study, 16 arthroscopy/sports medicine fellows and 2 senior residents (complete group: N = 18) were randomly assigned to perform a precourse cadaveric ABR (Bankart subgroup: N = 6), ARCR (cuff subgroup: N = 6), or basic skills on a shoulder simulator (N = 6). After completing a PBP training curriculum, all 18 registrants performed both an ABR and ARCR scored in real time by trained raters using previously validated metrics. RESULTS: The Bankart subgroup made 58% fewer objectively assessed errors at the completion of the course than at baseline (P = .004, confidence interval -1.449 to -0.281), and performance variability was substantially reduced (standard deviation = 5.89 vs 2.81). The cuff subgroup also made 58% fewer errors (P = .001, confidence interval -1.376 to 0.382) and showed a similar reduction in performance variability (standard deviation = 5.42 vs 2.1). Only one subject's precourse baseline performance met the proficiency benchmark compared with 89% and 83% of the all registrants on the final ABR and ARCR cadaveric assessments, respectively. CONCLUSIONS: The results of this study reject the null hypothesis. They demonstrate that the implementation of a PBP simulation curriculum to train the skills necessary to perform arthroscopic Bankart and rotator cuff repairs results in a large and statistically significant improvement in the trainee's ability to meet the 2 related performance benchmarks. Proficiency was demonstrated by 89% and 83% of the trainees for an ABR and an ARCR, respectively, in a two- and one-half day course. CLINICAL RELEVANCE: Surgical training employing a PBP curriculum is efficient, effective, and has the potential to improve patient safety.

Arthroscopic Rotator Cuff Repair Metrics: Establishing Face, Content, and Construct Validity in a Cadaveric Model.

PURPOSE: To create and determine face validity and content validity of arthroscopic rotator cuff repair (ARCR) performance metrics, to confirm construct validity of the metrics coupled with a cadaveric shoulder, and to establish a performance benchmark for the procedure on a cadaveric shoulder. METHODS: Five experienced arthroscopic shoulder surgeons created step, error, and sentinel error metrics for an ARCR. Fourteen shoulder arthroscopy faculty members from the Arthroscopy Association of North America formed the modified Delphi panel to assess face and content validity. Eight Arthroscopy Association of North America shoulder arthroscopy faculty members (experienced group) were compared with 9 postgraduate year 4 or 5 orthopaedic residents (novice group) in their ability to perform an ARCR. Instructions were given to perform a diagnostic arthroscopy and a 2-anchor, 4-simple suture repair of a 2-cm supraspinatus tear. The procedure was videotaped in its entirety and independently scored in blinded fashion by trained, paired reviewers. RESULTS: Delphi panel consensus for 42 steps and 66 potential errors was obtained. Overall performance assessment showed a mean inter-rater reliability of 0.93. Novice surgeons completed 17% fewer steps (32.1 vs 37.5, P = .001) and enacted 2.5 times more errors than the experienced group (6.21 vs 2.5, P = .012). Fifty percent of the experienced group members and none of the novice group members achieved the proficiency benchmark of a minimum of 37 steps completed with 3 or fewer errors. CONCLUSIONS: Face validity and content validity for the ARCR metrics, along with construct validity for the metrics and cadaveric shoulder, were verified. A proficiency benchmark was established based on the mean performance of an experienced group of arthroscopic shoulder surgeons. CLINICAL RELEVANCE: Validated procedural metrics combined with the use of a cadaveric shoulder can be used to accurately assess the performance of an ARCR.

Editorial Commentary: Harnessing Surgical Simulation-Saddle up for a Long and Rigorous Ride!

Arthroscopic simulation tools will assist in the training of skills that are transferrable to the operating room. Success in that endeavor requires the following: relevant outcome metrics rather than surrogate measures of performance, evaluation tools that enable scoring of step and error metrics rather than global rating scales, construct validation of simulators and metrics based on actual performance, development of simulators that not only assess performance but also train additional skills and provide feedback, and a focus first on effective training as opposed to high-fidelity realism.

Surgical Management of Proximal Long Head Biceps Tendon Disorders.

Disorders of the long head of the biceps tendon can make a significant contribution to shoulder pain and dysfunction. Historically, open biceps tenotomy or a proximal tenodesis of the tendon through a deltopectoral approach was used to manage biceps tendonitis and instability. Recent developments have added additional options. Arthroscopic techniques offer minimally invasive methods to secure the biceps in the suprapectoral region at the appropriate length. An open subpectoral tenodesis provides for simple exposure that secures the tendon just proximal to the musculotendinous junction and eliminates potentially diseased tendon and synovium from the biceps groove. Common fixation methods include the use of interference screws, suture anchors, and button devices. Future developments will likely add modified tenotomy techniques that will minimize the Popeye deformity in low-demand individuals. Simpler, but secure methods of fixation for higher demand patients will minimize the operative time required.

Inter-rater Reliability for Metrics Scored in a Binary Fashion-Performance Assessment for an Arthroscopic Bankart Repair.

PURPOSE: To determine the inter-rater reliability (IRR) of a procedure-specific checklist scored in a binary fashion for the evaluation of surgical skill and whether it meets a minimum level of agreement (≥0.8 between 2 raters) required for high-stakes assessment. METHODS: In a prospective randomized and blinded fashion, and after detailed assessment training, 10 Arthroscopy Association of North America Master/Associate Master faculty arthroscopic surgeons (in 5 pairs) with an average of 21 years of surgical experience assessed the video-recorded 3-anchor arthroscopic Bankart repair performance of 44 postgraduate year 4 or 5 residents from 21 Accreditation Council for Graduate Medical Education orthopaedic residency training programs from across the United States. RESULTS: No paired scores of resident surgeon performance evaluated by the 5 teams of faculty assessors dropped below the 0.8 IRR level (mean = 0.93; range 0.84-0.99; standard deviation = 0.035). A comparison between the 5 assessor groups with 1 factor analysis of variance showed that there was no significant difference between the groups (P = .205). Pearson's product-moment correlation coefficient revealed a strong and statistically significant negative correlation, that is, -0.856 (P < .000), indicating that as intra-operative error rate scores increased, the IRR decreased. CONCLUSIONS: Arthroscopy Association of North America shoulder faculty raters from across the United States showed high levels of IRR in the assessment of an arthroscopic 3-anchor Bankart repair procedure. All paired assessments were above the 0.8 level and the mean IRR of all resident assessments was 0.93, indicating that they could be used for high-stakes decisions. CLINICAL RELEVANCE: With the move toward outcomes-based performance evaluation for graduate medical education, high-stakes assessments of surgical skill will require robust, reliable measurement tools that are able to withstand challenge. Surgical checklists employing metrics scored in a binary fashion meet the need and can show a high (>80%) IRR.

Editorial Commentary: Healthy Skepticism: An Invaluable Trait.

Using a switching stick from the posterior portal is a frequently employed method of identifying the location for the subscapularis split for a shoulder arthroscopic Latarjet. However, in a recent cadaveric study, in none of the 20 specimens studied did this technique result in the split being placed in the preferred position-the junction of the superior two-thirds and the inferior one-third of the subscapularis. As a result, if the position of the coracoid graft is too superior, the anteroinferior extension of the glenoid arc and the vector of the conjoined sling may be suboptimal and fail to restrain humeral head translation. There is thus a need for a more reliable method for establishing the subscapularis split in patients with shoulder instability and significant glenoid bone loss. Moreover, in comparison to the open Latarjet surgical technique, an arthroscopic Latarjet procedure is notably more complex, technically challenging, and involves a demanding learning curve. For any procedure, the morbidity curve is inversely related to the learning curve-the lower one is on the learning curve, the greater the expected complications. The area beneath the morbidity curve is related to increased costs to manage complications and suboptimal outcomes. Study, practice, acquisition, and refinement of the necessary skills needed to perform a complex procedure should be done primarily in the cadaver lab. It is imperative that each of us critically evaluate our own experience and clinical practice to determine if we have or expect to generate the critical volume necessary to successfully master the arthroscopic Latarjet techniques and to retain the expertise needed for optimal patient outcomes.

Editorial Commentary: "A Brush Too Broad" (Validation, Global Rating Scales, and the Like What Do They Really Mean?).

Surgical simulation will play an increasingly important role in helping to train surgical skills. Tools of varying fidelity including benchtop models and virtual reality computer-generated simulations will be needed to most effectively train for accomplishing specific tasks and techniques. We must search for and identify how best to employ simulation to enable the trainee to master the requisite skills. It is imperative that in our investigation, validation methodologies, rating scales, and statistical tools be rigorously and accurately employed lest we be led astray by acceptance and assumption too freely given. The measure of our effectiveness will be whether or not patients enjoy improved outcomes and experience a reduction in surgical errors and complications.

Editorial Commentary: A Task May Be Done Quickly, but Not Necessarily Well.

The value of video gaming in relation to endoscopic surgical skills is explored in this article. Subjects who had more experience and performed better on 3-dimensional games also performed better on an arthroscopic simulator during tasks requiring manipulation of tools. Although the correlation is established, no data are presented to support causation. Video gaming holds promise in assisting the trainee attempting to develop components of endoscopic skill. Further investigation is required to demonstrate that improvements in performance can be expected to result.

A Proficiency-Based Progression Training Curriculum Coupled With a Model Simulator Results in the Acquisition of a Superior Arthroscopic Bankart Skill Set.

PURPOSE: To determine the effectiveness of proficiency-based progression (PBP) training using simulation both compared with the same training without proficiency requirements and compared with a traditional resident course for learning to perform an arthroscopic Bankart repair (ABR). METHODS: In a prospective, randomized, blinded study, 44 postgraduate year 4 or 5 orthopaedic residents from 21 Accreditation Council for Graduate Medical Education-approved US orthopaedic residency programs were randomly assigned to 1 of 3 skills training protocols for learning to perform an ABR: group A, traditional (routine Arthroscopy Association of North America Resident Course) (control, n = 14); group B, simulator (modified curriculum adding a shoulder model simulator) (n = 14); or group C, PBP (PBP plus the simulator) (n = 16). At the completion of training, all subjects performed a 3 suture anchor ABR on a cadaveric shoulder, which was videotaped and scored in blinded fashion with the use of previously validated metrics. RESULTS: The PBP-trained group (group C) made 56% fewer objectively assessed errors than the traditionally trained group (group A) (P = .011) and 41% fewer than group B (P = .049) (both comparisons were statistically significant). The proficiency benchmark was achieved on the final repair by 68.7% of participants in group C compared with 36.7% in group B and 28.6% in group A. When compared with group A, group B participants were 1.4 times, group C participants were 5.5 times, and group C(PBP) participants (who met all intermediate proficiency benchmarks) were 7.5 times as likely to achieve the final proficiency benchmark. CONCLUSIONS: A PBP training curriculum and protocol coupled with the use of a shoulder model simulator and previously validated metrics produces a superior arthroscopic Bankart skill set when compared with traditional and simulator-enhanced training methods. CLINICAL RELEVANCE: Surgical training combining PBP and a simulator is efficient and effective. Patient safety could be improved if surgical trainees participated in PBP training using a simulator before treating surgical patients.

The Bankart Performance Metrics Combined With a Cadaveric Shoulder Create a Precise and Accurate Assessment Tool for Measuring Surgeon Skill.

PURPOSE: To determine if previously validated performance metrics for an arthroscopic Bankart repair (ABR) coupled with a cadaveric shoulder are a valid assessment tool with the ability to discriminate between the performances of experienced and novice surgeons and to establish a proficiency benchmark for an ABR using a cadaveric shoulder. METHODS: Ten master/associate master faculty from an Arthroscopy Association of North America Resident Course (experienced group) were compared with 12 postgraduate year 4 and postgraduate year 5 orthopaedic residents (novice group). Each group was instructed to perform a diagnostic arthroscopy and a 3 suture anchor Bankart repair on a cadaveric shoulder. The procedure was videotaped in its entirety and independently scored in blinded fashion by a pair of trained reviewers. Scoring was based on defined and previously validated metrics for an ABR and included steps, errors, "sentinel" (more serious) errors, and time. RESULTS: The inter-rater reliability was 0.92. Novice surgeons made 50% more errors (5.86 v 2.95, P = .013), showed more performance variability (SD, 1.86 v 0.55), and took longer to perform the procedure (45.5 minutes v 25.9 minutes, P < .001). The greatest difference in errors related to suture delivery and management (exclusive of knot tying) (1.95 v 0.45, P = .024). CONCLUSIONS: The assessment tool composed of validated arthroscopic Bankart metrics coupled with a cadaveric shoulder accurately distinguishes the performance of experienced from novice orthopaedic surgeons. A benchmark based on the mean performance of the experienced group includes completion of a 3-anchor Bankart repair, and enacting no more than 3 total errors and 1 sentinel error. CLINICAL RELEVANCE: Validated procedural metrics combined with the use of a cadaveric shoulder can be used to assess the performance of an ABR. The methodology used may serve as a template for outcomes-based procedural skills training in general.

Objective Assessment of Knot-Tying Proficiency With the Fundamentals of Arthroscopic Surgery Training Program Workstation and Knot Tester.

PURPOSE: To assess a new method for biomechanical assessment of arthroscopic knots and to establish proficiency benchmarks using the Fundamentals of Arthroscopic Surgery Training (FAST) Program workstation and knot tester. METHODS: The first study group included 20 faculty at an Arthroscopy Association of North America resident arthroscopy course (19.9 ± 8.25 years in practice). The second group comprised 30 experienced surgeons attending an Arthroscopy Association of North America fall course (17.1 ± 19.3 years in practice). The training group included 44 postgraduate year 4 or 5 orthopaedic residents in a randomized, prospective study of proficiency-based training, with 3 subgroups: group A, standard training (n = 14); group B, workstation practice (n = 14); and group C, proficiency-based progression using the knot tester (n = 16). Each subject tied 5 arthroscopic knots backed up by 3 reversed hitches on alternating posts. Knots were tied under video control around a metal mandrel through a cannula within an opaque dome (FAST workstation). Each suture loop was stressed statically at 15 lb for 15 seconds. A calibrated sizer measured loop expansion. Knot failure was defined as 3 mm of loop expansion or greater. RESULTS: In the faculty group, 24% of knots "failed" under load. Performance was inconsistent: 12 faculty had all knots pass, whereas 2 had all knots fail. In the second group of practicing surgeons, 21% of the knots failed under load. Overall, 56 of 250 knots (22%) tied by experienced surgeons failed. For the postgraduate year 4 or 5 residents, the aggregate knot failure rate was 26% for the 220 knots tied. Group C residents had an 11% knot failure rate (half the overall faculty rate, P = .013). CONCLUSIONS: The FAST workstation and knot tester offer a simple and reproducible educational approach for enhancement of arthroscopic knot-tying skills. Our data suggest that there is significant room for improvement in the quality and consistency of these important arthroscopic skills, even for experienced arthroscopic surgeons. LEVEL OF EVIDENCE: Level II, prospective comparative study.

Metric Development for an Arthroscopic Bankart Procedure: Assessment of Face and Content Validity.

PURPOSE: To establish the metrics (operational definitions) necessary to characterize a reference arthroscopic Bankart procedure, and to seek consensus from experienced shoulder arthroscopists on the appropriateness of the steps, as well as errors identified. METHODS: Three experienced arthroscopic shoulder surgeons and an experimental psychologist (comprising the Metrics Group) deconstructed an arthroscopic Bankart procedure. Fourteen full-length videos were analyzed to identify the essential steps and potential errors. Sentinel (i.e., more serious) errors were defined as either (1) potentially jeopardizing the procedure outcome or (2) creating iatrogenic damage to the shoulder. The metrics were stress tested for clarity and the ability to be scored in binary fashion during a video review as either occurring or not occurring. The metrics were subjected to analysis by a panel of 27 experienced arthroscopic shoulder surgeons to obtain face and content validity using a modified Delphi Panel methodology (consensus opinion of experienced surgeons rendered by cyclical deliberations). RESULTS: Forty-five steps and 13 phases characterizing an arthroscopic Bankart procedure were identified. Seventy-seven procedural errors were specified, with 20 designated as sentinel errors. The modified Delphi Panel deliberation created the following changes: 2 metrics were deleted, 1 was added, and 5 were modified. Consensus on the resulting Bankart metrics was obtained and face and content validity verified. CONCLUSIONS: This study confirms that a core group of experienced arthroscopic surgeons is able to perform task deconstruction of an arthroscopic Bankart repair and create unambiguous step and error definitions (metrics) that accurately characterize the essential components of the procedure. Analysis and revision by a larger panel of experienced arthroscopists were able to validate the Bankart metrics. CLINICAL RELEVANCE: The ability to perform task deconstruction and validate the resulting metrics will play a key role in improving surgical skills training and assessing trainee progression toward proficiency.