Developing expertise in gynecologic surgery: reflective perspectives of international experts on learning environments and processes.

Research in medical education does not provide a clear understanding of how professional expertise develops among surgeons and what experiential factors contribute to that development. To address this gap, the researchers interviewed 16 international experts in female pelvic medicine and reconstructive surgery to assess their reflective perceptions of what specific opportunities and experiences initiated and supported their development toward expertise in their field. Characteristics and influences explaining the speed and quality of expertise development were sorted into the following themes: the dynamic process of expertise development, internal and personal characteristics, general aptitudes and preparatory skills, role modeling and interpersonal influences, opportunities to learn and practice, and roles and reference points. Across the narratives and perspectives of these expert surgeons, both individual characteristics and choices, and contextual activities and opportunities were necessary and important. Experiences with greatest impact on quality of expertise development included those provided by the environment and mentors, as well as those sought out by learners themselves, to elaborate and supplement existing opportunities. The ideal combination across experts was interaction and integration of individual characteristics with experiential opportunities. Grounded in theory and research in expertise development, these findings can support improvement of medical education, both for individual mentors and strategic program development. As surgery evolves at a continuously increasing pace, effective mentoring of promising surgical trainees will be critical to ensure that future generations of gynecologic surgeons will remain excellent. Effective, efficient surgical expertise development requires identifying trainees with the appropriate characteristics and providing them with the best development opportunities.

Defining Expertise in Gynecologic Surgery: Perspectives of Expert Gynecologic Surgeons.

OBJECTIVE: The aim of this study was to describe how professional expertise is defined and understood among gynecologic surgeons and what experiential factors contribute to that understanding. METHODS: Semistructured interviews with 16 experts in Female Pelvic Medicine and Reconstructive Surgery were conducted to identify how expertise in their field is defined, recognized, and assessed. Independent thematic analysis of the interview transcripts was performed by each member of the research team and then distilled and synthesized into convergent themes. RESULTS: Experts described surgical expertise as difficult to define but with several dominant themes including knowledge, technical skills, clinical experience, adaptability, continuous learning, communication, and professional recognition. Expertise requires judgment in applying technical skills to meet each patient's specific needs. Experts described unique ways of seeing and thinking during surgery, characterized by spatial awareness of relevant anatomy, temporal awareness of future changes, and rapidly adaptive application of their skills enabling them to do difficult tasks with fluidity, making the tasks seem easy to observers. These expert surgeons acknowledged that achieving expertise requires hard work and maintaining expertise requires continuous learning, highlighted by challenge seeking to do the most difficult tasks in their field. They also noted the importance of effective communication of their knowledge to others, which contributes to their perception as experts by colleagues in the field. CONCLUSIONS: Surgical expertise is a complex phenomenon with several meaningful themes. Understanding the authentic nature of surgical expertise can be used to support the development of competencies and the effective mentoring of promising surgical trainees to achieve surgical expertise.

Validity: applying current concepts and standards to gynecologic surgery performance assessments.

Validity is critical for meaningful assessment of surgical competency. According to the Standards for Educational and Psychological Testing, validation involves the integration of data from well-defined classifications of evidence. In the authoritative framework, data from all classifications support construct validity claims. The two aims of this study were to develop a categorization method for validity evidence published in support of surgery performance assessments and to summarize the results of applying this methodology to the gynecologic surgery literature. This was a critical analysis of published observations reported as validity evidence in studies with a construct validity claim. Medline and Embase databases were searched using keywords: "surgery" and "construct validity". Parameters included English-language articles published from 2000 to 2012. Gynecologic studies were analyzed for definitions of construct validity and nonstandard terminology. Categorization criteria were developed and applied by the researchers to all observations. Two independent evaluators examined reported observations for compliance with guidelines provided by the Standards. Inter-rater agreement was calculated using weighted kappa. The initial search returned 167 articles. Twenty-five articles were left for inclusion in our analysis. Eighteen (72 %) articles defined construct validity as the ability to discriminate between expert and novice levels of proficiency. Within the sample, 80 discrete observations of reported validity evidence were identified and categorized according to standard classifications. Nearly 30 % of all published observations intended to demonstrate differences in performance by level of proficiency, 25 % described a scoring model, and 14 % demonstrated support of assessment content. Not one article contained a statistical correlation between assessment scores and objective outcomes from the authentic surgical environment. Medians for level of rigor ranged from 0 to 1 across all forms of evidence. Weighted kappa values ranged 0.60-0.91. Validity claims in gynecologic surgical assessment over-rely on generalizability evidence. No test-criterion evidence was observed. Increased awareness of current standards and systematic argument development is needed for gynecologic performance assessments.

Designing and evaluating a STEM teacher learning opportunity in the research university.

This study examines the design and evaluation strategies for a year-long teacher learning and development experience, including their effectiveness, efficiency and recommendations for strategic redesign. Design characteristics include programmatic features and outcomes: cognitive, affective and motivational processes; interpersonal and social development; and performance activities. Program participants were secondary math and science teachers, partnered with engineering faculty mentors, in a research university-based education and support program. Data from multiple sources demonstrated strengths and weaknesses in design of the program's learning environment, including: face-to-face and via digital tools; on-site and distance community interactions; and strategic evaluation tools and systems. Implications are considered for the strategic design and evaluation of similar grant-funded research experiences intended to support teacher learning, development and transfer.

Redesigning and aligning assessment and evaluation for a federally funded math and science teacher educational program.

This paper examines the redesign of evaluation components for a teacher professional development project funded by the National Science Foundation. It focuses on aligning evaluation instrumentation and strategies with program goals, research goals and program evaluation best practices. The study identifies weaknesses in the original (year 1) program evaluation design and implementation, develops strategies and tracks changes for year 2 implementation, and then reports enhancement of findings and recommendations for year 3. It includes lessons learned about assessment and evaluation over the project lifespan, with implications for research and evaluation of a range of related programs. This study functions as a classic illustration of how critical it is to observe first principles of assessment and evaluation for funded programs, the risks that arise when they are ignored, and the benefits that accrue when they are systematically observed.