Video-based interventions to improve self-assessment accuracy among physicians: A systematic review.

PURPOSE: Self-assessment of a physician's performance in both procedure and non-procedural activities can be used to identify their deficiencies to allow for appropriate corrective measures. Physicians are inaccurate in their self-assessments, which may compromise opportunities for self- development. To improve this accuracy, video-based interventions of physicians watching their own performance, an experts' performance or both, have been proposed to inform their self-assessment. We conducted a systematic review of the effectiveness of video-based interventions targeting improved self-assessment accuracy among physicians. MATERIALS AND METHODS: The authors performed a systematic search of MEDLINE, Embase, EBM reviews, and Scopus databases from inception to August 23, 2022, using combinations of terms for "self-assessment", "video-recording", and "physician". Eligible studies were empirical investigations assessing the effect of video-based interventions on physicians' self-assessment accuracy with a comparison of self-assessment accuracy pre- and post- video intervention. We defined self-assessment accuracy as a "direct comparison between an external evaluator and self-assessment that was quantified using formal statistical analysis". Two reviewers independently screened records, extracted data, assessed risk of bias, and evaluated quality of evidence. A narrative synthesis was conducted, as variable outcomes precluded a meta-analysis. RESULTS: A total of 2,376 papers were initially retrieved. Of these, 22 papers were selected for full-text review; a final 9 studies met inclusion criteria for data extraction. Across studies, 240 participants from 5 specialties were represented. Video-based interventions included self-video review (8/9), benchmark video review (3/9), and/or a combination of both types (1/9). Five out of nine studies reported that participants had inaccurate self-assessment at baseline. After the intervention, 5 of 9 studies found a statistically significant improvement in self-assessment accuracy. CONCLUSIONS: Overall, current data suggests video-based interventions can improve self-assessment accuracy. Benchmark video review may enable physicians to improve self-assessment accuracy, especially for those with limited experience performing a particular clinical skill. In contrast, self-video review may be able to provide improvement in self-assessment accuracy for more experience physicians. Future research should use standardized methods of comparison for self-assessment accuracy, such as the Bland-Altman analysis, to facilitate meta-analytic summation.

Accuracy of self-assessment in gastrointestinal endoscopy: a systematic review and meta-analysis.

BACKGROUND: Assessment is necessary to ensure both attainment and maintenance of competency in gastrointestinal (GI) endoscopy, and this can be accomplished through self-assessment. We conducted a systematic review with meta-analysis to evaluate the accuracy of self-assessment among GI endoscopists. METHODS: This was an individual participant data meta-analysis of studies that investigated self-assessment of endoscopic competency. We performed a systematic search of the following databases: Ovid MEDLINE, Ovid EMBASE, Wiley Cochrane CENTRAL, and ProQuest Education Resources Information Center. We included studies if they were primary investigations of self-assessment accuracy in GI endoscopy that used statistical analyses to determine accuracy. We conducted a meta-analysis of studies using a limits of agreement (LoA) approach to meta-analysis of Bland-Altman studies. RESULTS: After removing duplicate entries, we screened 7138 records. After full-text review, we included 16 studies for qualitative analysis and three for meta-analysis. In the meta-analysis, we found that the LoA were wide (-41.0 % to 34.0 %) and beyond the clinically acceptable difference. Subgroup analyses found that both novice and intermediate endoscopists had wide LoA (-45.0 % to 35.1 % and -54.7 % to 46.5 %, respectively) and expert endoscopists had narrow LoA (-14.2 % to 21.4 %). CONCLUSIONS: GI endoscopists are inaccurate in self-assessment of their endoscopic competency. Subgroup analyses demonstrated that novice and intermediate endoscopists were inaccurate, while expert endoscopists have accurate self-assessment. While we advise against the sole use of self-assessment among novice and intermediate endoscopists, expert endoscopists may wish to integrate it into their practice.

Impact of a simulation-based ergonomics training curriculum on work-related musculoskeletal injury risk in colonoscopy.

BACKGROUND AND AIMS: Endoscopists are at risk of developing musculoskeletal injuries (MSIs), and few receive training on ergonomics. The aim of this study was to determine the impact of a simulation-based ergonomics training curriculum (ETC) on work-related MSI risk during clinical colonoscopy. METHODS: Novice endoscopists underwent a simulation-based ETC and were compared with an historical control group who received simulation-based training without ergonomics training. The ETC included a didactic lecture and video on ergonomics in colonoscopy, feedback from supervisors on ergonomics, and an ergonomics checklist to augment feedback and promote self-reflection. Participants were assessed using the rapid entire body assessment (REBA) and rapid upper limb assessment (RULA). The primary outcome was participants' REBA scores during 2 clinical colonoscopies 4 to 6 weeks after training. RESULTS: In clinical colonoscopy, the ETC group had superior REBA scores (clinical procedure 1: median score, 6 vs 11; P < .001; clinical procedure 2: median score, 6 vs 10; P < .001). In a simulated colonoscopy, the ETC group did not have significantly different REBA or RULA scores between baseline, immediately after training, and 4 to 6 weeks after (REBA: median scores of 5, 5, and 5, respectively; P > .05; RULA: median scores of 6, 6, and 6, respectively; P > .05). The control group had worsening REBA and RULA scores during the study timeline (REBA: median scores of 5 at baseline, 9 immediately after training, and 9 at 4-6 weeks after training; P < .001; RULA: median scores of 6, 7, and 7, respectively; P = .04) during simulated procedures. CONCLUSIONS: A simulation-based ETC is associated with reduced risk of MSI during endoscopy. Although the REBA score was improved, the intervention group was still within the medium-risk range.

Non-technical skills curriculum incorporating simulation-based training improves performance in colonoscopy among novice endoscopists: Randomized controlled trial.

BACKGROUND AND AIMS: Non-technical skills (NTS), involving cognitive, social and interpersonal skills that complement technical skills, are important for the completion of safe and efficient procedures. We investigated the impact of a simulation-based curriculum with dedicated NTS training on novice endoscopists' performance of clinical colonoscopies. METHODS: A single-blinded randomized controlled trial was conducted at a single center. Novice endoscopists were randomized to a control curriculum or a NTS curriculum. The control curriculum involved a didactic session, virtual reality (VR) simulator colonoscopy training, and integrated scenario practice using a VR simulator, a standardized patient, and endoscopy nurse. Feedback and training were provided by experienced endoscopists. The NTS curriculum group received similar training that included a small-group session on NTS, feedback targeting NTS, and access to a self-reflective NTS checklist. The primary outcome was performance during two clinical colonoscopies, assessed using the Joint Advisory Group Direct Observation of Procedural Skills (JAG DOPS) tool. RESULTS: Thirty-nine participants completed the study. The NTS group (n = 21) had superior clinical performance during their first (P < 0.001) and second clinical colonoscopies (P < .0.001), compared to the control group (n = 18). The NTS group performed significantly better on the VR simulator (P < 0.05) and in the integrated scenario (P < 0.05). CONCLUSION: Our findings demonstrate that dedicated NTS training led to improved performance of clinical colonoscopies among novices.

Undisclosed payments by pharmaceutical and medical device manufacturers to authors of endoscopy guidelines in the United States.

BACKGROUND AND AIMS: Payments from pharmaceutical and medical device manufacturers to authors of clinical practice guidelines (CPGs) may influence practice recommendations. It is therefore important to evaluate the completeness of financial conflict of interest (FCOI) declarations among CPG authors. METHODS: We performed a cross-sectional analysis of industry payments to authors of endoscopy guidelines published by 5 GI societies between 2014 and 2017. For each author we identified payments using the disclosure section of CPGs and the Centers for Medicare & Medicaid Services Open Payments (CMS-OP) database. We calculated the prevalence, monetary value, and type of declared and undeclared payments among authors. Payments were assessed for the calendar year of and before publication. RESULTS: Thirty-seven CPGs were included in the analysis comprising 569 author entries (91 unique individuals; 66.43% men, 92.6% physicians, 66.4% academically affiliated). Four hundred fifty-one episodes (79%) involved FCOIs, 451 (79%) had undisclosed FCOIs in the CMS-OP, and 445 (77%) had FCOIs relevant to a CPG recommendation. The median undisclosed payment value was $4807.26 (interquartile range, $334.84-$20,579.75). Male authors (odds ratio, 2.23; 95% confidence interval, 1.47-3.39) and academically affiliated authors (odds ratio, 8.87; 95% confidence interval, 5.57-14.13) were significantly more likely to have undeclared payments (P < .001). No CPGs met all National Academy of Medicine criteria. CONCLUSIONS: Recognizing concerns about the accuracy of the CMS-OP, there are substantial discrepancies between industry-reported payments and author self-disclosure. Additionally, there is a high prevalence of undisclosed payments by pharmaceutical and medical device manufacturers to these authors. Given the potential impact of these discrepancies and undisclosed payments on CPGs, more accurate reporting and alternative strategies for managing FCOI are needed.