Decisions and Consequences: Validation of High-Stakes Simulation-Based Assessments in Gynecologic Surgery.

STUDY OBJECTIVE: Simulation-based assessment is poised for application in educational promotion and credentialing in gynecologic surgery. With high-stakes assessment, validation necessitates evidence, not just about the trustworthiness of decisions but of beneficial consequences to education and healthcare as well. In this paper, we unpack the modern conceptualization of validity as it pertains to surgical simulation and high-stakes competency assessment. DESIGN: N/A SETTING: N/A PATIENTS: N/A INTERVENTIONS: N/A MEASUREMENTS AND MAIN RESULTS: N/A CONCLUSION: Validity in high-stakes simulation-based assessment necessitates evidence, not just about the trustworthiness of score-based decisions but of beneficial consequences to education and healthcare as well.

A Systematic Review of Validity Evidence for the Fundamentals of Laparoscopic Surgery Examination in Gynecology.

OBJECTIVE: The Fundamentals in Laparoscopic Surgery (FLS) examination is designed to test laparoscopic surgery skills. Our aim for this systematic review was to examine validity evidence supporting or refuting the FLS examination specifically as a high-stakes summative assessment tool in gynecology. DATA SOURCES: The data sources were PubMed, MEDLINE, Embase, and Scopus. METHODS OF STUDY SELECTION: The study eligibility criterion was the subject of the FLS examination as an assessment tool in gynecology. We developed a data extraction tool and assigned articles for screening and extraction to all authors, who then abstracted data independently. Conflicts that arose during the extraction process were resolved by consensus. We organized validity evidence for the cognitive and manual skills portions on the basis of the categories of current validation standards. TABULATION, INTEGRATION, AND RESULTS: From 1971 citations identified, 9 studies were included, involving 319 participants. For the cognitive portion of the test, the results were mixed in 5 studies in relationships with the other variables category. For the manual portion of the test, most of the studies focused on the relationships with other variables evidence with mixed findings. The concerning findings in the manual skills portion included the lack of transferability of skills to the operating room, limited mixed evidence for improvement in operating room performance, and worse performance by obstetrics and gynecology surgeons compared with other specialties. We did not find supportive content-based, response process, or consequential evidence in either the cognitive or manual skills portion of the test. CONCLUSION: Validity evidence for the FLS examination was either mixed, as it pertained to relationships with other variables, or lacking in other important evidence categories. Further evidence is required to justify the use of the FLS examination scores as a high-stakes summative assessment.

Opioid Pain Medication Use in New Urogynecology Patients.

OBJECTIVES: The aim of this study was to determine the prevalence of opioid pain medication use among patients presenting for a new visit to the urogynecology clinic compared with those presenting to general gynecology. METHODS: We identified all patients who presented for new patient visits to the urogynecology and gynecology clinics between January 1, 2016, and December 31, 2016. Any previous or current opioid use was extracted from the electronic medical record medication list. Statistical analysis was performed using χ and Fisher exact tests for comparisons of categorical variables. Modified Poisson regression models were used to estimate prevalence proportion ratios (PPRs). RESULTS: There were 1835 (955 gynecology, 880 urogynecology) patients included. Median age was 47 years (interquartile range, 29 years), and median body mass index was 28.15 kg/m (interquartile range, 9.96 kg/m). Prevalence of opioid use was lowest among women who identified as Asian or other race and highest among black and Native American women; however, when compared by ethnicity, use was lowest among Hispanic women (P = 0.01). Among new urogynecology patients, 14% had self-reported opioid pain medication usage. Opioid use was almost twice as likely in the urogynecology group (PPR, 1.86; 95% confidence interval, 1.4-2.4). When adjusted for confounders, the urogynecology group was 1.3 times as likely to report opioid use (PPR 1.29; 95% confidence interval, 1.0-1.8), with this result approaching statistical significance. CONCLUSIONS: Opioid use is greater in patients presenting to the urogynecology clinic compared with general gynecology. Urogynecologists need to know this information for planning and optimizing pain management in this population.

Randomized controlled trial of belladonna and opiate suppository during intravesical onabotulinum toxin A injection.

BACKGROUND: Intravesical injection with onabotulinum toxin A injection can be performed in-office under local anesthesia. Rectally administered pain medication presents a potentially feasible and previously uninvestigated adjunct to office-based anesthesia protocols. OBJECTIVE: The primary aim of this study was to determine whether adding a belladonna and opiate suppository to standard lidocaine instillation resulted in reduction of bladder injection pain during onabotulinum toxin A injection procedure. STUDY DESIGN: This was a prospective, randomized, double-blind, placebo-controlled study of patients undergoing onabotulinum toxin A bladder injection at a single clinic. Patients age ≥18 years, who met clinical criteria for invasive treatment of refractory urinary symptoms, had previously documented postvoid residual volumes <150 mL, and elected for in-office intravesical onabotulinum toxin A injection were eligible to participate. Participants were randomized by computer-generated block randomization to receive a belladonna and opiate (belladonna alkaloid with morphine 16.2/7.5 mg) or placebo suppository. Suppositories were placed immediately prior to lidocaine-based anesthesia, which all participants received. All participants underwent a standardized injection procedure using the same rigid cystoscope, needle type, and injection pattern (20 injections total). A 0-10 numeric rating scale was used to assess pain intensity before anesthesia and suppository, 40 minutes after administration of anesthesia and suppository, after first 10 bladder injections, and immediately after completion of 20 injections. Pain increase during procedure was calculated using the difference between score 40 minutes after administration of anesthesia and suppository and score after first 10 bladder injections. Postvoid residual were measured immediately postprocedure and 2 weeks later. Patient satisfaction with pain control was measured using a Likert scale. Our primary outcome was change in pain level from anesthetic baseline to midprocedure (score after first 10 bladder injections to score 40 minutes after administration of anesthesia and suppository). A final sample size of 26 patients was needed to have 80% power (alpha = 0.05) to detect a 50% reduction in bladder injection pain during the procedure as defined by our primary outcome. An intent-to-treat approach was used for all analyses. RESULTS: In all, 26 participants were enrolled and randomized with 13 in each study arm. Participants in the treatment group were slightly older than in the placebo group (P = .05); there were no statistically significant differences in medical comorbidities. Median score after first 10 bladder injections to score 40 minutes after administration of anesthesia and suppository for the placebo group and treatment group was 4 (range 1-10) and 5 (range 0,9), respectively (P = .94). Median scores immediately after completion of 20 injections for the placebo group and treatment group were 3 (range 0-10) and 2 (range 0,8), respectively (P = .29). There were no significant differences in preinjection pain scores reported before anesthesia and suppository and at 40 minutes after administration of anesthesia and suppository. Postprocedure postvoid residual >200 mL was noted in 5 (38%) of the placebo group and 3 (23%) of the treatment group (P = .67). Two-week postprocedure postvoid residual >200 mL was noted in 3 (25%) of the placebo group and 2 (15%) of the treatment group (P = .64) for an overall rate of 20%. Eleven (84%) participants in each group reported being "mostly satisfied" or "very much satisfied" with pain control. CONCLUSION: Belladonna and opiate suppository use did not significantly reduce bladder injection pain, or increase risk of urinary retention immediately postprocedure or 2 weeks later. Satisfaction with pain control among onabotulinum toxin A injection patients is high.

Developing expertise in gynecologic surgery: reflective perspectives of international experts on learning environments and processes.

Research in medical education does not provide a clear understanding of how professional expertise develops among surgeons and what experiential factors contribute to that development. To address this gap, the researchers interviewed 16 international experts in female pelvic medicine and reconstructive surgery to assess their reflective perceptions of what specific opportunities and experiences initiated and supported their development toward expertise in their field. Characteristics and influences explaining the speed and quality of expertise development were sorted into the following themes: the dynamic process of expertise development, internal and personal characteristics, general aptitudes and preparatory skills, role modeling and interpersonal influences, opportunities to learn and practice, and roles and reference points. Across the narratives and perspectives of these expert surgeons, both individual characteristics and choices, and contextual activities and opportunities were necessary and important. Experiences with greatest impact on quality of expertise development included those provided by the environment and mentors, as well as those sought out by learners themselves, to elaborate and supplement existing opportunities. The ideal combination across experts was interaction and integration of individual characteristics with experiential opportunities. Grounded in theory and research in expertise development, these findings can support improvement of medical education, both for individual mentors and strategic program development. As surgery evolves at a continuously increasing pace, effective mentoring of promising surgical trainees will be critical to ensure that future generations of gynecologic surgeons will remain excellent. Effective, efficient surgical expertise development requires identifying trainees with the appropriate characteristics and providing them with the best development opportunities.

Defining Expertise in Gynecologic Surgery: Perspectives of Expert Gynecologic Surgeons.

OBJECTIVE: The aim of this study was to describe how professional expertise is defined and understood among gynecologic surgeons and what experiential factors contribute to that understanding. METHODS: Semistructured interviews with 16 experts in Female Pelvic Medicine and Reconstructive Surgery were conducted to identify how expertise in their field is defined, recognized, and assessed. Independent thematic analysis of the interview transcripts was performed by each member of the research team and then distilled and synthesized into convergent themes. RESULTS: Experts described surgical expertise as difficult to define but with several dominant themes including knowledge, technical skills, clinical experience, adaptability, continuous learning, communication, and professional recognition. Expertise requires judgment in applying technical skills to meet each patient's specific needs. Experts described unique ways of seeing and thinking during surgery, characterized by spatial awareness of relevant anatomy, temporal awareness of future changes, and rapidly adaptive application of their skills enabling them to do difficult tasks with fluidity, making the tasks seem easy to observers. These expert surgeons acknowledged that achieving expertise requires hard work and maintaining expertise requires continuous learning, highlighted by challenge seeking to do the most difficult tasks in their field. They also noted the importance of effective communication of their knowledge to others, which contributes to their perception as experts by colleagues in the field. CONCLUSIONS: Surgical expertise is a complex phenomenon with several meaningful themes. Understanding the authentic nature of surgical expertise can be used to support the development of competencies and the effective mentoring of promising surgical trainees to achieve surgical expertise.

Validity: applying current concepts and standards to gynecologic surgery performance assessments.

Validity is critical for meaningful assessment of surgical competency. According to the Standards for Educational and Psychological Testing, validation involves the integration of data from well-defined classifications of evidence. In the authoritative framework, data from all classifications support construct validity claims. The two aims of this study were to develop a categorization method for validity evidence published in support of surgery performance assessments and to summarize the results of applying this methodology to the gynecologic surgery literature. This was a critical analysis of published observations reported as validity evidence in studies with a construct validity claim. Medline and Embase databases were searched using keywords: "surgery" and "construct validity". Parameters included English-language articles published from 2000 to 2012. Gynecologic studies were analyzed for definitions of construct validity and nonstandard terminology. Categorization criteria were developed and applied by the researchers to all observations. Two independent evaluators examined reported observations for compliance with guidelines provided by the Standards. Inter-rater agreement was calculated using weighted kappa. The initial search returned 167 articles. Twenty-five articles were left for inclusion in our analysis. Eighteen (72 %) articles defined construct validity as the ability to discriminate between expert and novice levels of proficiency. Within the sample, 80 discrete observations of reported validity evidence were identified and categorized according to standard classifications. Nearly 30 % of all published observations intended to demonstrate differences in performance by level of proficiency, 25 % described a scoring model, and 14 % demonstrated support of assessment content. Not one article contained a statistical correlation between assessment scores and objective outcomes from the authentic surgical environment. Medians for level of rigor ranged from 0 to 1 across all forms of evidence. Weighted kappa values ranged 0.60-0.91. Validity claims in gynecologic surgical assessment over-rely on generalizability evidence. No test-criterion evidence was observed. Increased awareness of current standards and systematic argument development is needed for gynecologic performance assessments.

Is de novo stress incontinence after sacrocolpopexy related to anatomical changes and surgical approach?

INTRODUCTION AND HYPOTHESIS: The objective was to investigate the relationship between new onset postoperative stress urinary incontinence (SUI) after sacrocolpopexy (SCP) and anatomical change/surgical approach. METHODS: We analyzed a retrospective cohort of patients with negative preoperative testing for SUI who underwent SCP from 2005 to 2012. Our primary outcome was new onset postoperative SUI. Logistic regression was used to examine the relationship among anatomical change, defined as ΔAa, ΔBa, ΔC, and ΔTVL, and surgical approach, categorized as abdominal (ASCP) for open cases and minimally invasive (MISCP) for laparoscopic and robot-assisted cases, and postoperative SUI. RESULTS: Of 795 cases, 33 ASCP (43%) and 44 MISCP (57%) met the inclusion criteria for analysis. New onset SUI was demonstrated by 15 patients (45%) of the ASCP group and 7 patients (15%) of the MISCP group (p = 0.005). New onset SUI was significantly associated with route of SCP and ΔAa (p = 0.006 and p = 0.033 respectively). Controlling for ΔAa, the odds of new onset SUI were 4.4 times higher in the ASCP group compared with the MISCP group (OR 4.37, 95% CI 1.42, 13.48). Controlling for route of SCP, the odds of new onset SUI were 2.2 times higher with moderate ΔAa compared with low ΔAa (OR 2.16 95% CI 1.07, 4.38). The odds of new onset SUI was 4.7 times higher in those with high ΔAa than in those with low ΔAa (OR 4.67 95% CI 1.14, 19.22). ΔBa, ΔC, and ΔTVL were not associated with new onset SUI. CONCLUSIONS: Greater reduction in point Aa and abdominal surgical route are risk factors for new onset postoperative SUI after SCP.

Venous thromboembolism in women undergoing pelvic reconstructive surgery with mechanical prophylaxis alone.

INTRODUCTION AND HYPOTHESIS: The objective of this study was determine the frequency of symptomatic perioperative venous thromboembolism (VTE) and risk factor(s) associated with VTE occurrence in women undergoing elective pelvic reconstructive surgery using only intermittent pneumatic compression (IPC) for VTE prophylaxis. METHODS: A multi-center case-cohort retrospective review was conducted at six clinical sites over a 66-month period. All sites utilize IPC as standard VTE prophylaxis for urogynecological surgery. VTE cases occurring during the same hospitalization and up to 6 weeks postoperatively were identified by ICD9 code query. Four controls were temporally matched to each case. Information collected included demographics, medical history, route of surgery, operative time, and intraoperative characteristics. Univariate and multivariate backward stepwise logistic regression analyses were performed to identify potential risk factors for VTE. RESULTS: Symptomatic perioperative VTE was diagnosed in 27 subjects from a cohort of 10,627 women who underwent elective urogynecological surgery (0.25 %). Univariate analysis identified surgical route (laparotomy vs others), type of surgery ("major" vs "minor"), history of gynecological cancer, surgery time, and patient age as risk factors for VTE (P < 0.05). Multivariate analysis identified increased frequency of VTE with laparotomy, age ≥ 70, and surgery duration ≥ 5 h. CONCLUSIONS: In our study cohort, the frequency of symptomatic perioperative VTE was low. Laparotomy, age ≥ 70 years, and surgery duration ≥ 5 h were associated with VTE occurrence.

Appearance of the levator ani muscle subdivisions in endovaginal three-dimensional ultrasonography.

OBJECTIVE: To formulate an evaluation system and authenticate the levator ani subdivisions visible on endovaginal three-dimensional ultrasonography. METHODS: A three-dimensional endovaginal ultrasound examination was performed on five fresh-frozen female pelves. The location of the pubovaginalis, puboperinealis, puboanalis, puborectalis, and iliococcygeus muscles was identified. Three-dimensional endovaginal ultrasound scans of 22 nulliparous women with normal pelvic floors were obtained, and a three-level evaluation system was constructed. Level 1 contained the muscles as they insert into the perineal body. Level 2 evaluated the insertions into the retropubic area and typically contained the pubovaginalis, puboperinealis, puboanalis, and puborectalis. Level 3 contained muscles that were cephalad to the pubic bone. Two blinded reviewers assessed levator ani subdivision visualization at each level and individual muscle visualization by the origin-insertion points. RESULTS: There was 98%, 96%, and 92% agreement for levels 1, 2, and 3 muscles, with 95% confidence intervals (CIs) of 0.92-1, 0.95-0.99, and 0.88-0.95, respectively. Kappa values (95% CI) for agreement were calculated for individual muscles as follows: superficial transverse perinei and puborectalis were seen by both raters 100%, puboperinealis, pubovaginalis, and puboanalis 0.645 (0.1-1), and iliococcygeus 0.9 (0.6-1). CONCLUSION: The location of the pubovaginalis, puboperinealis, puboanalis, puborectalis, and iliococcygeus as seen by three-dimensional endovaginal ultrasonography was confirmed through anatomic dissection of fresh-frozen pelves. Subdivisions of the levator ani muscle were visualized reliably with three-dimensional ultrasonography using a systematic approach. LEVEL OF EVIDENCE: III.