Comparison of dyad versus individual simulation-based training on stress, anxiety, cognitive load, and performance: a randomized controlled trial.

BACKGROUND: Dyad learning has been shown to be an effective tool for teaching procedural skills, but little is known about how dyad learning may impact the stress, anxiety, and cognitive load that a student experiences when learning in this manner. In this pilot study, we investigate the relationship between dyad training on stress, anxiety, cognitive load, and performance in a simulated bradycardia scenario. METHODS: Forty-one fourth-year medical school trainees were randomized as dyads (n = 24) or individuals (n = 17) for an education session on day 1. Reassessment occurred on day 4 and was completed as individuals for all trainees. Primary outcomes were cognitive load (Paas scale), stress (Cognitive Appraisal Ratio), and anxiety levels (abbreviated State-Trait Anxiety Inventory). Secondary outcomes were time-based performance metrics. RESULTS: On day 1 we observed significant differences for change in anxiety and stress measured before and after the training scenario between groups. Individuals compared to dyads had larger mean increases in anxiety, (19.6 versus 7.6 on 80-point scale, p = 0.02) and stress ratio (1.8 versus 0.9, p = 0.045). On the day 4 post-intervention assessment, no significant differences were observed between groups. Secondary outcomes were significant for shorter time to diagnosis of bradycardia (p = 0.01) and time to initiation of pacing (p = 0.04) in the dyad group on day 1. On day 4, only time to recognizing the indication for pacing was significantly shorter for individual training (hazard ratio [HR] = 2.26, p = 0.02). CONCLUSIONS: Dyad training results in lower stress and anxiety levels with similar performance compared to individual training.

Personalized video feedback improves suturing skills of incoming general surgery trainees.

BACKGROUND: The American Board of Surgery encourages graduating medical students to prepare for surgical residency before day 1. We sought to determine the impact of personalized video feedback on an advance preparation task. METHODS: We conducted a nonrandomized study comparing video feedback versus no feedback. We sent incoming surgical interns a preparatory package 2 months before starting residency. Trainees video-recorded themselves performing a subcuticular wound closure, 3 times at 3-week intervals, and submitted these for appraisal. A staff surgeon provided personalized feedback on each video as a narrated voiceover. The voiced-over videos were then returned to trainees. We compared performance (time and completion rate) on suturing in a multistation assessment against residents from the previous year (no-feedback group). RESULTS: The feedback group had a higher completion rate for the suturing assessment than the no-feedback group (23/28 [82%] vs. 8/27 [30%], P < .0001). The feedback group also completed the suturing station at a faster rate than those without feedback (hazard ratio 4.9 [95% confidence interval (CI): 2.2,11.2), P < .0001). Global rating scores were significantly higher for the feedback group (mean difference [5-point scale] = 0.7 [95% CI: 0.3, 1.1]). However, Objective Structured Assessment of Technical Skills scores indicated no significant difference between groups (mean difference [5-point scale] = 0.3 [95% CI: 0.0, 0.6]). Within the feedback group, we found significant improvement from baseline to final performances (mean difference = 109 seconds [95% CI: 79, 140]). CONCLUSION: Personalized narrated feedback as part of a home-based advance preparation package for incoming residents is associated with higher performance on early objective assessments.

Surgery Website as a 24/7 Adjunct to a Surgical Curriculum.

OBJECTIVE: Successfully teaching duty hour restricted trainees demands engaging learning opportunities. Our surgical educational website and its associated assets were assessed to understand how such a resource was being used. DESIGN: Our website was accessible to all Mayo Clinic employees via the internal web network. Website access data from April 2015 through October 2016 were retrospectively collected using Piwik. SETTING: Academic, tertiary care referral center with a large general surgery training program. Mayo Clinic, Rochester, MN. PARTICIPANTS: A total of 257 Mayo Clinic employees used the website. RESULTS: The website had 48,794 views from 6313 visits by 257 users who spent an average of 14 ± 11 minutes on the website. Our website houses 295 videos, 51 interactive modules, 14 educational documents, and 7 flashcard tutorials. The most popular content type was videos, with a total of 30,864 views. The most popular visiting time of the day was between 8 pm and 9 pm with 6358 views (13%), and Thursday was the most popular day with 17,907 views (37%).  A total of 78% of users accessed content beyond the homepage. Average visits peaked in relation to 2 components of our curriculum: a 240% increase one day before our biannual intern simulation assessments, and a 61% increase one day before our weekly conducted Friday simulation sessions. Interns who rotated on the service of the staff surgeon who actively endorses the website had 93% more actions per visit as compared to other users. The highest clicks were on the home banner for our weekly simulation session pre-emptive videos, followed by "groin anatomy," and "TEP hernia repair" videos. CONCLUSIONS: Our website acted as a "just-in-time" accessible portal to reliable surgical information. It supplemented the time sensitive educational needs of our learners by serving as a heavily used adjunct to 3 components of our surgical education curriculum: weekly simulation sessions, biannual assessments, and clinical rotations.

Trends in P Value, Confidence Interval, and Power Analysis Reporting in Health Professions Education Research Reports: A Systematic Appraisal.

PURPOSE: To characterize reporting of P values, confidence intervals (CIs), and statistical power in health professions education research (HPER) through manual and computerized analysis of published research reports. METHOD: The authors searched PubMed, Embase, and CINAHL in May 2016, for comparative research studies. For manual analysis of abstracts and main texts, they randomly sampled 250 HPER reports published in 1985, 1995, 2005, and 2015, and 100 biomedical research reports published in 1985 and 2015. Automated computerized analysis of abstracts included all HPER reports published 1970-2015. RESULTS: In the 2015 HPER sample, P values were reported in 69/100 abstracts and 94 main texts. CIs were reported in 6 abstracts and 22 main texts. Most P values (≥77%) were ≤.05. Across all years, 60/164 two-group HPER studies had ≥80% power to detect a between-group difference of 0.5 standard deviations. From 1985 to 2015, the proportion of HPER abstracts reporting a CI did not change significantly (odds ratio [OR] 2.87; 95% CI 1.04, 7.88) whereas that of main texts reporting a CI increased (OR 1.96; 95% CI 1.39, 2.78). Comparison with biomedical studies revealed similar reporting of P values, but more frequent use of CIs in biomedicine. Automated analysis of 56,440 HPER abstracts found 14,867 (26.3%) reporting a P value, 3,024 (5.4%) reporting a CI, and increased reporting of P values and CIs from 1970 to 2015. CONCLUSIONS: P values are ubiquitous in HPER, CIs are rarely reported, and most studies are underpowered. Most reported P values would be considered statistically significant.

Personalized Video Feedback and Repeated Task Practice Improve Laparoscopic Knot-Tying Skills: Two Controlled Trials.

PURPOSE: Compare the effect of personalized feedback (PF) vs. task demonstration (TD), both delivered via video, on laparoscopic knot-tying skills and perceived workload; and evaluate the effect of repeated practice. METHOD: General surgery interns and research fellows completed four repetitions of a simulated laparoscopic knot-tying task at one-month intervals. Midway between repetitions, participants received via e-mail either a TD video (demonstration by an expert) or a PF video (video of their own performance with voiceover from a blinded senior surgeon). Each participant received at least one video per format, with sequence randomly assigned. Outcomes included performance scores and NASA Task Load Index (NASA-TLX) scores. To evaluate the effectiveness of repeated practice, scores from these trainees on a separate delayed retention test were compared against historical controls who did not have scheduled repetitions. RESULTS: Twenty-one trainees completed the randomized study. Mean change in performance scores was significantly greater for those receiving PF (difference = 23.1 of 150 [95% confidence interval (CI): 0, 46.2], P = .05). Perceived workload was also significantly reduced (difference = -3.0 of 20 [95% CI: -5.8, -0.3], P = .04). Compared with historical controls (N = 93), the 21 with scheduled repeated practice had higher scores on the laparoscopic knot-tying assessment two weeks after the final repetition (difference = 1.5 of 10 [95% CI: 0.2, 2.8], P = .02). CONCLUSIONS: Personalized video feedback improves trainees' procedural performance and perceived workload compared with a task demonstration video. Brief monthly practice sessions support skill acquisition and retention.

The ACGME Case Log System May Not Accurately Represent Operative Experience Among General Surgery Interns.

OBJECTIVE: To assess if the Accreditation Council for Graduate Medical Education (ACGME) case log system accurately captures operative experience of our postgraduate year 1 (PGY-1) residents. DESIGN: ACGME case log information was retrospectively obtained for 5 cohorts of PGY-1 residents (2011-2015) and compared to the number of operative cases captured by an institutional automated operative case report system, Surgical Access Utility System (SAUS). SAUS automatically captures all surgical team members who are listed in the operative dictation for a given case, including interns. A paired t-test analysis was used to compare number of cases coded between the 2 systems. SETTING: Academic, tertiary care referral center with a large general surgery training program. PARTICIPANTS: PGY-1 general surgery trainees (interns) from the years 2011-2015. RESULTS: Forty-nine PGY-1 general surgery residents were identified over a 5-year period. Mean operative case volume per intern, per year, captured by the automated SAUS was 176.5 ± 28.1 (SD) compared to 126.3 ± 58.0 ACGME cases logged (mean difference = 50.2 cases, p < 0.001). CONCLUSIONS: ACGME case log data may not accurately reflect the actual operative experience of our PGY-1 residents. If such data holds true for other general surgery training programs, the true impact of duty hour regulations on operative volume may be unclear when using the ACGME case log data. This current standard approach for using ACGME case logs as a representation of operative experience requires further scrutiny and potential revision to more accurately determine operative experience for accreditation purposes.

At Home Preresidency Preparation for General Surgery Internship: A Pilot Study.

OBJECTIVE: To create a novel "at-home" preresidency preparatory adjunct for medical students entering surgical residency. DESIGN: Preparatory resources were mailed to match medical students before residency matriculation in 2015. This included "how-to" videos, low-cost models, and surgical instruments for 5 "stations" (arterial blood gas analysis, anatomy and imaging knowledge, knot tying ability, and suturing dexterity) of our program's biannual general surgery intern objective assessment activity (Surgical Olympics: total 13 stations, 10 points each). Scores from 2015 were compared with 2014 historical controls in a retrospective manner using the Student's t-test. SETTING: Academic, tertiary care referral center with a large general surgery training program. PARTICIPANTS: Postgraduate year 1 general surgery trainees (interns) from the years 2014 and 2015. RESULTS: Twenty-six interns participated in the 2015 assessment and were compared to thirty-two 2014 interns. Overall mean scores were low, but higher (19.7 vs. 15.4, p = 0.04) in the 2015 class. The largest increase was noted in the anatomy knowledge station (mean = 5.0 vs. 1.9, p < 0.01). Scores in stations assessing technical competence were similar to controls. The number of perfect scores among the 5 stations was higher (10 vs. 5) in the 2015 group. Mean scores from the other 8 stations, for which no resources were mailed, showed no difference (29.3 vs. 28.3, p = 0.75). CONCLUSIONS: Enacting a simple, home-based curriculum for medical students before surgical residency, improved performance on early knowledge assessments.