Assessment of colonoscopy skill using machine learning to measure quality: Proof-of-concept and initial validation.

Background and study aims Low-quality colonoscopy increases cancer risk but measuring quality remains challenging. We developed an automated, interactive assessment of colonoscopy quality (AI-CQ) using machine learning (ML). Methods Based on quality guidelines, metrics selected for AI development included insertion time (IT), withdrawal time (WT), polyp detection rate (PDR), and polyps per colonoscopy (PPC). Two novel metrics were also developed: HQ-WT (time during withdrawal with clear image) and WT-PT (withdrawal time subtracting polypectomy time). The model was pre-trained using a self-supervised vision transformer on unlabeled colonoscopy images and then finetuned for multi-label classification on another mutually exclusive colonoscopy image dataset. A timeline of video predictions and metric calculations were presented to clinicians in addition to the raw video using a web-based application. The model was externally validated using 50 colonoscopies at a second hospital. Results The AI-CQ accuracy to identify cecal intubation was 88%. IT ( P = 0.99) and WT ( P = 0.99) were highly correlated between manual and AI-CQ measurements with a median difference of 1.5 seconds and 4.5 seconds, respectively. AI-CQ PDR did not significantly differ from manual PDR (47.6% versus 45.5%, P = 0.66). Retroflexion was correctly identified in 95.2% and number of right colon evaluations in 100% of colonoscopies. HQ-WT was 45.9% of, and significantly correlated with ( P = 0.85) WT time. Conclusions An interactive AI assessment of colonoscopy skill can automatically assess quality. We propose that this tool can be utilized to rapidly identify and train providers in need of remediation.

Incidence, Predictors, and Outcomes of Clinically Significant Post-ERCP Bleeding: a Contemporary Multi-center Study.

BACKGROUND AIMS: Clinically significant post-endoscopic retrograde cholangiopancreatography (ERCP) bleeding (CSPEB) is common. Contemporary estimates of risk are lacking. We aimed to identify risk factors for and outcomes following CSPEB. METHODS: We analyzed multi-center prospective ERCP data between 2018-2023 with 30-day follow-up. The primary outcome was CSPEB, defined as hematemesis, melena, or hematochezia resulting in: hemoglobin drop ≥20 g/L or transfusion and/or endoscopy to evaluate suspected bleeding, and/or unplanned healthcare visitation and/or prolongation of existing admission. Firth logistic regression was employed. P-values <0.05 were significant, with odds ratios (ORs) and 95% confidence intervals reported. RESULTS: CSPEB occurred following 129 (1.5%) of 8,517 ERCPs (mean onset 3.2 days), with 110 of 4,849 events (2.3%) occurring following higher-risk interventions (sphincterotomy, sphincteroplasty, pre-cut sphincterotomy, and/or needle-knife access). CSPEB patients required endoscopy and transfusion in 86.0% and 53.5% of cases, respectively, with three cases (2.3%) being fatal. P2Y12 inhibitors were held for a median of 4 days (IQR 4) prior to higher-risk ERCP. Following higher-risk interventions, P2Y12 inhibitors (OR 3.33, 1.26-7.74), warfarin (OR 8.54, 3.32-19.81), dabigatran (OR 13.40, 2.06-59.96), rivaroxaban (OR 7.42, 3.43-15.24) and apixaban (OR 4.16, 1.99-8.20) were associated with CSPEB. Significant intraprocedural bleeding post sphincterotomy (OR 2.32, 1.06-4.60), but not post sphincteroplasty, was also associated. Concomitant cardiorespiratory events occurred more frequently within 30 days following CSPEB (OR 12.71, 4.75-32.54). CONCLUSIONS: Risks of antiplatelet-related CSPEB may be underestimated by endoscopists based on observations of suboptimal holding before higher-risk ERCP. Appropriate periprocedural antithrombotic management is essential and could represent novel quality initiative targets.

Development of American Society for Gastrointestinal Endoscopy standards for training in advanced endoscopy within dedicated advanced endoscopy fellowship programs.

BACKGROUND AND AIMS: Training in interventional endoscopy is offered by nonaccredited advanced endoscopy fellowship programs (AEFPs). The number of these programs has increased dramatically with a concurrent increase in the breadth and complexity of interventional endoscopy procedures. Accreditation is governed by competency-based education, yet what constitutes a "high-quality" nonaccredited AEFP has not been defined. Using an evidence-based consensus process, we aimed to establish standards for AEFPs. METHODS: The RAND UCLA appropriateness method, a well-described modified Delphi process to develop quality indicators, was used. A task force established by the American Society for Gastrointestinal Endoscopy drafted potential quality indicators (structure, process, and outcome) in 6 categories: activity preceding training; structure of AEFPs; training in ERCP, EUS, and EMR; and luminal stent placement. Three rounds of iterative feedback from 20 experts were conducted. Round 0 involved discussion of project details. In round 1, experts independently ranked proposed quality indicators on a 9-point interval scale ranging from highly inappropriate (1) to highly appropriate (9). Next, proposed quality indicators were discussed and reworded in a group meeting followed by round 2, in which experts independently reranked proposed quality indicators and provided benchmarks (when applicable). The median score for each quality indicator was calculated. Mean absolute deviation from the median was calculated, and appropriateness of potential quality indicators was assessed using the BIOMED concerted action on appropriateness definition, P value method, and interpercentile range adjusted for symmetry definition. A quality indicator was deemed appropriate if the median score was ≥7 and met criteria for appropriateness using all 3 defined statistical methods. RESULTS: Of 89 proposed quality indicators, 37 statements met criteria as appropriate for a quality indicator (activity preceding training, 2; structure of AEFPs, 10; training in ERCP, 7; training in EUS, 8; training in EMR, 7; luminal stent placement, 3). Minimum thresholds were defined for 19 relevant quality indicators for number of trainers, procedures during fellowship, and procedures before assessment of competence. Among the final appropriate quality indicators were that all trainees should undergo qualitative and quantitative competence assessments using validated tools at least quarterly with documented feedback throughout the training period and that trainees should track outcomes and relevant quality metrics for specific procedures. CONCLUSIONS: This consensus process using validated methodology established standards for an AEFP in an effort to ensure adequate training in the most commonly taught interventional endoscopic procedures (ERCP, EUS, EMR, and luminal stent placement) during fellowship. An important component of an AEFP is the use of competency-based assessments that are compliant with the Accreditation Council for Graduate Medical Education's Next Accreditation System, with the goal of ensuring that trainees achieve specific milestones in their progression to achieving cognitive and technical competency.

Consensus statements on the current landscape of artificial intelligence applications in endoscopy, addressing roadblocks, and advancing artificial intelligence in gastroenterology.

BACKGROUND AND AIMS: The American Society for Gastrointestinal Endoscopy (ASGE) AI Task Force along with experts in endoscopy, technology space, regulatory authorities, and other medical subspecialties initiated a consensus process that analyzed the current literature, highlighted potential areas, and outlined the necessary research in artificial intelligence (AI) to allow a clearer understanding of AI as it pertains to endoscopy currently. METHODS: A modified Delphi process was used to develop these consensus statements. RESULTS: Statement 1: Current advances in AI allow for the development of AI-based algorithms that can be applied to endoscopy to augment endoscopist performance in detection and characterization of endoscopic lesions. Statement 2: Computer vision-based algorithms provide opportunities to redefine quality metrics in endoscopy using AI, which can be standardized and can reduce subjectivity in reporting quality metrics. Natural language processing-based algorithms can help with the data abstraction needed for reporting current quality metrics in GI endoscopy effortlessly. Statement 3: AI technologies can support smart endoscopy suites, which may help optimize workflows in the endoscopy suite, including automated documentation. Statement 4: Using AI and machine learning helps in predictive modeling, diagnosis, and prognostication. High-quality data with multidimensionality are needed for risk prediction, prognostication of specific clinical conditions, and their outcomes when using machine learning methods. Statement 5: Big data and cloud-based tools can help advance clinical research in gastroenterology. Multimodal data are key to understanding the maximal extent of the disease state and unlocking treatment options. Statement 6: Understanding how to evaluate AI algorithms in the gastroenterology literature and clinical trials is important for gastroenterologists, trainees, and researchers, and hence education efforts by GI societies are needed. Statement 7: Several challenges regarding integrating AI solutions into the clinical practice of endoscopy exist, including understanding the role of human-AI interaction. Transparency, interpretability, and explainability of AI algorithms play a key role in their clinical adoption in GI endoscopy. Developing appropriate AI governance, data procurement, and tools needed for the AI lifecycle are critical for the successful implementation of AI into clinical practice. Statement 8: For payment of AI in endoscopy, a thorough evaluation of the potential value proposition for AI systems may help guide purchasing decisions in endoscopy. Reliable cost-effectiveness studies to guide reimbursement are needed. Statement 9: Relevant clinical outcomes and performance metrics for AI in gastroenterology are currently not well defined. To improve the quality and interpretability of research in the field, steps need to be taken to define these evidence standards. Statement 10: A balanced view of AI technologies and active collaboration between the medical technology industry, computer scientists, gastroenterologists, and researchers are critical for the meaningful advancement of AI in gastroenterology. CONCLUSIONS: The consensus process led by the ASGE AI Task Force and experts from various disciplines has shed light on the potential of AI in endoscopy and gastroenterology. AI-based algorithms have shown promise in augmenting endoscopist performance, redefining quality metrics, optimizing workflows, and aiding in predictive modeling and diagnosis. However, challenges remain in evaluating AI algorithms, ensuring transparency and interpretability, addressing governance and data procurement, determining payment models, defining relevant clinical outcomes, and fostering collaboration between stakeholders. Addressing these challenges while maintaining a balanced perspective is crucial for the meaningful advancement of AI in gastroenterology.

The Bethesda ERCP Skills Assessment Tool (BESAT) can reliably differentiate endoscopists of different experience levels.

Background and study aims The Bethesda ERCP Skill Assessment Tool (BESAT) is a video-based assessment tool of technical endoscopic retrograde cholangiopancreatography (ERCP) skill with previously established validity evidence. We aimed to assess the discriminative validity of the BESAT in differentiating ERCP skill levels. Methods Twelve experienced ERCP practitioners from tertiary academic centers were asked to blindly rate 43 ERCP videos using the BESAT. ERCP videos consisted of native biliary cannulation and sphincterotomy and were recorded from 10 unique endoscopists of various ERCP experience (from advanced endoscopy fellow to > 10 years of ERCP experience). Inter-rater reliability, discriminative validity, and internal structure validity were subsequently assessed. Results The BESAT was found to reliably differentiate between endoscopists of varying levels of ERCP experience with experienced ERCPists scoring higher than novice ERCPists in 11 of 13 (85%) instrument items. Inter-rater reliability for BESAT items ranged from good to excellent (intraclass correlation range: 0.86 to 0.93). Internal structure validity was assessed with item-total correlations ranging from 0.53 to 0.83. Conclusions Study findings demonstrate that the BESAT, a video-based ERCP skill assessment tool, has high inter-rater reliability and has discriminative validity in differentiating novice from expert ERCP skill. Further investigations are needed to determine the role of video-based assessment in improving trainee learning curves and patient outcomes.

Safety and feasibility of establishing an adjuvant hepatic artery infusion program.

BACKGROUND: Hepatic artery infusion (HAI) is less frequently used in the adjuvant setting for resectable colorectal liver metastasis (CRLM) due to concerns regarding toxicity. Our objective was to evaluate the safety and feasibility of establishing an adjuvant HAI program. METHODS: Patients who underwent HAI pump placement between January 2019 and February 2023 for CRLM were identified. Complications and HAI delivery were compared between patients who received HAI in the unresectable and adjuvant settings. RESULTS: Of 51 patients, 23 received HAI for unresectable CRLM and 28 in the adjuvant setting. Patients with unresectable CRLM more commonly had bilobar disease (n = 23/23 vs n = 18/28, p < 0.01) and more preoperative liver metastases (median 10 [IQR 6-15] vs 4 [IQR 3-7], p < 0.01). Biliary sclerosis was the most common complication (n = 2/23 vs n = 4/28); however, there were no differences in postoperative or HAI-specific complications. In the most recent two years, 0 patients in the unresectable group vs 2 patients in the adjuvant group developed biliary sclerosis. All patients were initiated on HAI with no difference in treatment times or dose reductions. CONCLUSION: Adjuvant HAI is safe and feasible for patients with resectable CRLM. HAI programs can carefully consider including patients with resectable CRLM if managed by an experienced multidisciplinary team with quality assurance controls in place.

Effect of an online educational module incorporating real-time feedback on accuracy of polyp sizing in trainees: a randomized controlled trial.

BACKGROUND: Although polyp size dictates surveillance intervals, endoscopists often estimate polyp size inaccurately. We hypothesized that an intervention providing didactic instruction and real-time feedback could significantly improve polyp size classification. METHODS: We conducted a multicenter randomized controlled trial to evaluate the impact of different components of an online educational module on polyp sizing. Participants were randomized to control (no video, no feedback), video only, feedback only, or video + feedback. The primary outcome was accuracy of polyp size classification into clinically relevant categories (diminutive [1-5mm], small [6-9mm], large [≥10mm]). Secondary outcomes included accuracy of exact polyp size (inmm), learning curves, and directionality of inaccuracy (over- vs. underestimation). RESULTS: 36 trainees from five training programs provided 1360 polyp size assessments. The feedback only (80.1%, P=0.01) and video + feedback (78.9%, P=0.02) groups had higher accuracy of polyp size classification compared with controls (71.6%). There was no significant difference in accuracy between the video only group (74.4%) and controls (P=0.42). Groups receiving feedback had higher accuracy of exact polyp size (inmm) and higher peak learning curves. Polyps were more likely to be overestimated than underestimated, and 29.3% of size inaccuracies impacted recommended surveillance intervals. CONCLUSIONS: Our online educational module significantly improved polyp size classification. Real-time feedback appeared to be a critical component in improving accuracy. This scalable and no-cost educational module could significantly decrease under- and overutilization of colonoscopy, improving patient outcomes while increasing colonoscopy access.

Limited exposure to pancreatic ERCP during advanced endoscopy training impacts competence and performance in independent practice.

BACKGROUND/OBJECTIVES: The impact of competency-based training programs on pancreatic endoscopic retrograde cholangiopancreatography (ERCP) performance remains unclear. This study aimed to describe the learning curves of pancreatic ERCP and subsequent performance during independent practice. METHODS: This was a multicenter prospective cohort study involving advanced endoscopy trainees (AETs). In the 1st phase, trainees were assessed on every 5th ERCP using the ERCP and EUS Skills Assessment Tool (TEESAT). Cumulative sum (CUSUM) analysis of pancreatic ERCP evaluations was used to establish learning curves. During the 2nd phase (1st year of independent practice), now-graduated participants documented their performance on key ERCP quality indicators. RESULTS: A total of 24 AETs (20 training programs) received sufficient evaluations for CUSUM analysis. Pancreatic ERCP accounted for 14.6 % (196/1339) of all ERCPs evaluated with 45 % of pancreatic ERCPs carrying a Grade 3 level of complexity. A minority of AETs (16.7 %) performed enough pancreatic ERCPs to generate meaningful learning curves with no AETs achieving competence in pancreatic cannulation, sphincterotomy, or stone clearance during Phase 1. In Phase 2, a total of 3620 ERCPs were performed, of which 281 (7.8 %) were pancreatic ERCPs. While the overall pancreatic duct cannulation rate was 92.2 %, the native papilla pancreatic duct cannulation rate was 85.7 %, which was below the recommended 90 % threshold. CONCLUSIONS: Advanced endoscopy training offers a low level of exposure to pancreatic ERCP, which is mirrored in independent practice, highlighting the inadequate training in pancreatic ERCP. Given the complexity of pancreatic ERCP, novel strategies are warranted to improve training in pancreatic ERCP.

Effect of Polypectomy Simulation-Based Mastery Learning on Skill Retention Among Practicing Endoscopists.

PURPOSE: Practicing endoscopists frequently perform and teach screening colonoscopies and polypectomies, but there is no standardized method to train and assess physicians who perform polypectomy procedures. The authors created a polypectomy simulation-based mastery learning (SBML) curriculum and hypothesized that completion of the curriculum would lead to immediate improvement in polypectomy skills and skill retention at 6 and 12 months after training. METHOD: The authors performed a pretest-posttest cohort study with endoscopists who completed SBML and were randomized to follow-up at 6 or 12 months from May 2021 to August 2022. Participants underwent SBML training, including a pretest, a video lecture, deliberate practice, and a posttest. All learners were required to meet or exceed a minimum passing standard on a 17-item skills checklist before completing training and were randomized to follow-up at 6 or 12 months. The authors compared simulated polypectomy skills performance on the checklist from pretest to posttest and posttest to 6- or 12-month follow-up test. RESULTS: Twenty-four of 30 eligible participants (80.0%) completed the SBML intervention, and 20 of 24 (83.3%) completed follow-up testing. The minimum passing standard was set at 93% of checklist items correct. The pretest passing rate was 4 of 24 participants (16.7%) compared with 24 of 24 participants (100%) at posttest ( P < .001). There were no significant differences in passing rates from posttest to combined 6- and 12-month posttest in which 18 of 20 participants (90.0%) passed. CONCLUSIONS: Before training and despite years of clinical experience, practicing endoscopists demonstrated poor performance of polypectomy skills. SBML was an effective method for practicing endoscopists to acquire and maintain polypectomy skills during a 6- to 12-month period.

Varied Trainee Competence in Cold Snare Polypectomy: Results of the COMPLETE Randomized Controlled Trial.

INTRODUCTION: Cold snare polypectomy (CSP) is strongly recommended as the optimal technique for the complete removal of small polyps. Though significant variability in polypectomy technique and quality has been established, the learning curve and impact of targeted training on CSP are unknown. Video feedback has shown promise as an effective pedagogy to improve performance among surgical trainees. We aimed to compare CSP performance between trainees who received video-based feedback and those who received conventional apprentice-based concurrent feedback. We hypothesized that video-based feedback would accelerate competence. METHODS: We conducted a single-blinded, randomized controlled trial on competence for CSP of polyps <1 cm, comparing video-based feedback with conventional feedback. We randomly assigned deidentified consecutively recorded CSP videos to blinded raters to assess using the CSP Assessment Tool. We shared cumulative sum learning curves every 25 CSP with each trainee. The video feedback trainees also received biweekly individualized terminal feedback. Control trainees received conventional feedback during colonoscopy. The primary outcome was CSP competence. We also assessed competence across domains and change over polypectomy volume. RESULTS: We enrolled and randomized 22 trainees, 12 to video-based feedback and 10 to conventional feedback, and evaluated 2,339 CSP. The learning curve was long; 2 trainees (16.7%) in the video feedback achieved competence, after a mean of 135 polyps, and no one in the control ( P = 0.481) achieved competence. Overall and in all steps of CSP, a higher percentage of the video feedback group met competence, increasing 3% every 20 CSP ( P = 0.0004). DISCUSSION: Video feedback aided trainees to competence in CSP. However, the learning curve was long. Our findings strongly suggest that current training methods are not sufficient to support trainees to competency by the completion of their fellowship programs. The impact of new training methods, such as simulation-based mastery learning, should be assessed to determine whether such methods can result in achievement of competence at a faster rate; ClinicalTrials.gov : NCT03115008.

Esophageal and Esophagogastric Junction Cancers, Version 2.2023, NCCN Clinical Practice Guidelines in Oncology.

Cancers originating in the esophagus or esophagogastric junction constitute a major global health problem. Esophageal cancers are histologically classified as squamous cell carcinoma (SCC) or adenocarcinoma, which differ in their etiology, pathology, tumor location, therapeutics, and prognosis. In contrast to esophageal adenocarcinoma, which usually affects the lower esophagus, esophageal SCC is more likely to localize at or higher than the tracheal bifurcation. Systemic therapy can provide palliation, improved survival, and enhanced quality of life in patients with locally advanced or metastatic disease. The implementation of biomarker testing, especially analysis of HER2 status, microsatellite instability status, and the expression of programmed death-ligand 1, has had a significant impact on clinical practice and patient care. Targeted therapies including trastuzumab, nivolumab, ipilimumab, and pembrolizumab have produced encouraging results in clinical trials for the treatment of patients with locally advanced or metastatic disease. Palliative management, which may include systemic therapy, chemoradiation, and/or best supportive care, is recommended for all patients with unresectable or metastatic cancer. Multidisciplinary team management is essential for all patients with locally advanced esophageal or esophagogastric junction cancers. This selection from the NCCN Guidelines for Esophageal and Esophagogastric Junction Cancers focuses on the management of recurrent or metastatic disease.

Validity evidence for observational ERCP competency assessment tools: a systematic review.

BACKGROUND : Assessment of competence in endoscopic retrograde cholangiopancreatography (ERCP) is critical for supporting learning and documenting attainment of skill. Validity evidence supporting ERCP observational assessment tools has not been systematically evaluated. METHODS : We conducted a systematic search using electronic databases and hand-searching from inception until August 2021 for studies evaluating observational assessment tools of ERCP performance. We used a unified validity framework to characterize validity evidence from five sources: content, response process, internal structure, relations to other variables, and consequences. Each domain was assigned a score of 0-3 (maximum score 15). We assessed educational utility and methodological quality using the Accreditation Council for Graduate Medical Education framework and the Medical Education Research Quality Instrument, respectively. RESULTS : From 2769 records, we included 17 studies evaluating 7 assessment tools. Five tools were studied for clinical ERCP, one for simulated ERCP, and one for simulated and clinical ERCP. Validity evidence scores ranged from 2 to 12. The Bethesda ERCP Skills Assessment Tool (BESAT), ERCP Direct Observation of Procedural Skills Tool (ERCP DOPS), and The Endoscopic Ultrasound (EUS) and ERCP Skills Assessment Tool (TEESAT) had the strongest validity evidence, with scores of 10, 12, and 11, respectively. Regarding educational utility, most tools were easy to use and interpret, and required minimal additional resources. Overall methodological quality (maximum score 13.5) was strong, with scores ranging from 10 to 12.5. CONCLUSIONS : The BESAT, ERCP DOPS, and TEESAT had strong validity evidence compared with other assessments. Integrating tools into training may help drive learners' development and support competency decision making.

Interventions to improve the performance of upper GI endoscopy quality indicators.

The promotion of quality and best practices in gastroenterology and endoscopy is an ongoing effort. For upper GI endoscopy, quality indicators derived from clinical studies and expert consensus have been long established but remain variably obtained. To date, data on interventions aimed to improve these indicators are scarce. We systematically reviewed the literature to identify interventions and measures demonstrated to improve the performance of previously established upper endoscopy quality indicators. We also identified evidence gaps and opportunities for improvement in this area.

Quality Indicators for the Detection and Removal of Colorectal Polyps and Interventions to Improve Them.

Modifiable risk factors for postcolonoscopy colorectal cancer include suboptimal lesion detection (missed neoplasms) and inadequate lesion removal (incomplete polypectomy) during colonoscopy. Competent detection and removal of colorectal polyps are thus fundamental to ensuring adequate colonoscopy quality. Several well-researched quality metrics for polyp detection have been implemented into clinical practice, chief among these the adenoma detection rate. Less data are available on quality indicators for polyp removal, which currently include complete resection rates and skills assessment tools. This review summarizes the available literature on quality indicators for the detection and removal of colorectal polyps, as well as interventions to improve them.