Myth or Reality: Self-Assessment Is Central to Effective Curriculum in Anatomical Pathology Graduate Medical Education.

Self-assessment, a personal evaluation of one's professional attributes and abilities against a perceived norm, has frequently been cited as a necessary component of self-directed learning and the maintenance of competency within regulated health professions, including the medical professions. However, education research literature has consistently shown uninformed personal global assessment of performance to be inaccurate in a variety of contexts, and have limited value in a workplace-based curriculum. Incorporating known standards of performance with internal and external data on the performance improves a learner's ability to accurately self-assess. Selecting content suitable for self-assessment, providing explicit assessment standards, encouraging feedback-seeking behaviors, supporting a growth mindset, and providing quality feedback in a supportive context are all strategies that can support learner self-assessment, learner engagement in reflection, and action on feedback in Anatomical Pathology graduate medical education.

Practical solutions for implementation of Transition to Practice curricula in a competency-based medical education model.

BACKGROUND: Although transition from residency to practice represents a critical learning stage, there is a paucity of literature to inform local curriculum development and implementation. OBJECTIVES: To describe local curriculum development for Transition to Practice (TTP) for use within a competency-based medical education model, including important content and suitable teaching and assessment strategies. DESIGN: We reviewed the literature to construct a definition and develop initial curriculum content for TTP. We then gathered local residency program directors' views on TTP content, teaching, and assessment via online survey and an international educational conference workshop. RESULTS: We identified 21 important TTP content areas in the literature and analyzed 35 survey responses, representing 33 residency programs. Survey participants viewed Further sophistication of clinical skills, How to set up a practice, and Time management skills as the three most important content areas. Views on content importance varied by program. For teaching and assessment strategies, most respondents preferred: assessing what residents could do, providing real-life practice opportunities, and offering workplace-based assessments. CONCLUSIONS: TTP curricula implementation should reflect nationally set, specialty-specific curriculum elements; locally developed priority content; and assessment and teaching strategies. Individual learner needs and imminent practice context should guide faculty approaches to curriculum delivery.Résumé.

CONTEXTE: Bien que la transition de la résidence à la pratique représente une étape d'apprentissage essentielle, on retrouve peu de littérature pour informer le développement et l'implantation des cursus locaux. OBJECTIFS: Décrire le développement d'un cursus local de Transition vers la pratique (TVP) pour un modèle d'éducation médicale par compétences, incluant le contenu important et des stratégies d'enseignement et d'évaluation adaptées. CONCEPTION: Nous avons recensé les écrits afin d'établir une définition et de développer le contenu initial du cursus de TVP. Nous avons ensuite recueilli l'opinion des directions de programmes de résidence locaux sur le contenu, l'enseignement et l'évaluation du cursus TVP par le biais d'un sondage en ligne et un atelier offert dans le cadre d'une conférence pédagogique internationale. RÉSULTATS: Nous avons identifié 21 sujets importants pour la TVP dans la littérature et nous avons analysé les réponses de 35 sondages représentant 33 programmes de résidence. Les trois sujets considérés les plus importants par les participants au sondage étaient le Perfectionnement avancé des compétences cliniques; Comment débuter sa pratique médicale et les Aptitudes de gestion du temps. Les opinions sur l'importance du contenu variaient selon les programmes. Pour ce qui est des stratégies d'apprentissage et d'enseignement, la plupart des personnes sondées ont exprimé les préférences suivantes : évaluer ce que les résidents sont en mesure de faire; fournir des occasions réelles de pratiquer ; offrir des évaluations en milieu de travail. CONCLUSIONS: La mise en œuvre des cursus TVP devrait refléter des éléments des curriculums propres aux spécialités et définis à l'échelle nationale; des sujets prioritaires développés localement; des stratégies d'enseignement et d'évaluation. Les besoins personnels des étudiants et le contexte imminent du début de pratique devraient orienter les approches du corps professoral utilise dans l'enseignement de ce cursus.

Feedback Frequency in Competence by Design: A Quality Improvement Initiative.

BACKGROUND: Otolaryngology-head and neck surgery is in the first wave of residency training programs in Canada to adopt Competence by Design (CBD), a model of competency-based medical education. CBD is built on frequent, low-stakes assessments and requires an increase in the number of feedback interactions. The University of Toronto otolaryngology-head and neck surgery residents piloted the CBD model but were completing only 1 assessment every 4 weeks, which was insufficient to support CBD. OBJECTIVE: This project aimed to increase assessment completion to once per resident per week using quality improvement methodology. METHODS: Stakeholder engagement activities had residents and faculty characterize barriers to assessment completion. Brief electronic assessment forms were completed by faculty on residents' personal mobile devices in face-to-face encounters, and the number completed per resident was tracked for 10 months during the 2016-2017 pilot year. Response to the intervention was analyzed using statistical process control charts. RESULTS: The first bundled intervention-a rule set dictating which clinical instance should be assessed, combined with a weekly reminder implemented for 10 weeks-was unsuccessful in increasing the frequency of assessments. The second intervention was a leaderboard, designed on an audit-and-feedback system, which sent weekly comparison e-mails of each resident's completion rate to all residents and the program director. The leaderboard demonstrated significant improvement from baseline over 10 weeks, increasing the assessment completion rate from 0.22 to 2.87 assessments per resident per week. CONCLUSIONS: A resident-designed audit-and-feedback leaderboard system improved the frequency of CBD assessment completion.

Who's connected to whom and how: a model of evolving relationships and roles in faculty development and curriculum development during curriculum renewal and innovation.

This article was migrated. The article was marked as recommended. Faculty development and curriculum development are essential to the work of academic health sciences institutions. Through collegial conversations, more intense dialogue, and 'workshopping', we have identified a new model of how these two practices can be effectively integrated. We propose that this new model can create a system of knowledge mobilization and quality improvement that will greatly enhance curricular renewal and innovation. We invite and welcome comments and feedback from the health professions education community.

Exploring the experience of residents during the first six months of family medicine residency training.

BACKGROUND: The shift from undergraduate to postgraduate education signals a new phase in a doctor's training. This study explored the resident's perspective of how the transition from undergraduate to postgraduate (PGME) training is experienced in a Family Medicine program as they first meet the reality of feeling and having the responsibility as a doctor. METHODS: Qualitative methods explored resident experiences using interpretative inquiry through monthly, individual in-depth interviews with five incoming residents during the first six months of training. Focus groups were also held with residents at various stages of training to gather their reflection about their experience of the first six months. Residents were asked to describe their initial concerns, changes that occurred and the influences they attributed to those changes. RESULTS: Residents do not begin a Family Medicine PGME program knowing what it means to be a Family Physician, but learn what it means to fulfill this role. This process involves adjusting to significant shifts in responsibility in the areas of Knowledge, Practice Management, and Relationships as they become more responsible for care outcomes. CONCLUSION: This study illuminated the resident perspective of how the transition is experienced. This will assist medical educators to better understand the early training experiences of residents, how these experiences contribute to consolidating their new professional identity, and how to better align teaching strategies with resident learning needs.

Key-feature questions for assessment of clinical reasoning: a literature review.

OBJECTIVES: Key-feature questions (KFQs) have been developed to assess clinical reasoning skills. The purpose of this paper is to review the published evidence on the reliability and validity of KFQs to assess clinical reasoning. METHODS: A literature review was conducted by searching MEDLINE (1946-2012) and EMBASE (1980-2012) via OVID and ERIC. The following search terms were used: key feature; question or test or tests or testing or tested or exam; assess or evaluation, and case-based or case-specific. Articles not in English were eliminated. RESULTS: The literature search resulted in 560 articles. Duplicates were eliminated, as were articles that were not relevant; nine articles that contained reliability or validity data remained. A review of the references and of citations of these articles resulted in an additional 12 articles to give a total of 21 for this review. Format, language and scoring of KFQ examinations have been studied and modified to maximise reliability. Internal consistency reliability has been reported as being between 0.49 and 0.95. Face and content validity have been shown to be moderate to high. Construct validity has been shown to be good using vector thinking processes and novice versus expert paradigms, and to discriminate between teaching methods. The very modest correlations between KFQ examinations and more general knowledge-based examinations point to differing roles for each. Importantly, the results of KFQ examinations have been shown to successfully predict future physician performance, including patient outcomes. CONCLUSIONS: Although it is inaccurate to conclude that any testing format is universally reliable or valid, published research supports the use of examinations using KFQs to assess clinical reasoning. The review identifies areas of further study, including all categories of evidence. Investigation into how examinations using KFQs integrate with other methods in a system of assessment is needed.

Remediation of residents in difficulty: a retrospective 10-year review of the experience of a postgraduate board of examiners.

PURPOSE: To determine, through a 10-year review, (1) the prevalence of residents in difficulty, (2) characteristics of these residents, (3) areas of residents' weakness, and (4) outcomes of residents who undergo remediation. METHOD: A retrospective review of resident records for the University of Toronto Faculty of Medicine's (UT-FOM) Board of Examiners for Postgraduate Programs (BOE-PG) was done from July 1, 1999 to June 30, 2009 using predetermined data elements entered into a standardized form and analyzed for trends and significance. Outcomes for residents in difficulty were tracked through university registration systems and licensure databases. RESULTS: During 10 years, 103 UT-FOM residents were referred to the BOE-PG, representing 3% of all residents enrolled. The annual prevalence of residents referred to the BOE-PG ranged from 0.2% to 1.5%. The CanMEDS framework was used to classify areas of residents' weaknesses and organize remediation plans. All 100 residents studied had either medical expertise (85%) or professionalism (15%) weaknesses or both. Residents had difficulties with an average of 2.6 CanMEDS Roles, with highest frequencies of Medical Expert (85%) Professional (51%), Communicator (49%), Manager (43%), and Collaborator (20%). Often, there were multiple remediation periods, with an average of six months' duration. Usually, remediation was successful; 78% completed residency education, 17% were unsuccessful, and 5% remained in training. CONCLUSION: Residents in difficulty have multiple areas of weakness. The CanMEDS framework is an effective approach to classifying problems and designing remediation plans. Successful completion of residency education after remediation is the most common outcome.

Competency-based education in family medicine.

BACKGROUND: As a way of demonstrating an objective assessment of trainee competence, the College of Family Physicians of Canada has recently approved a competency-based framework known as CanMEDS-FM. All training programs in family medicine in Canada will be required to demonstrate the development of curriculum and evaluation methods based on the roles defined by the framework. AIM: This article describes the rationale and the approach used to develop a competency-based education curriculum in the postgraduate family medicine program at the University of Toronto. METHOD: The authors describe a systematic approach to curriculum development which includes the formation of a central steering committee, content development by faculty experts, mapping of curriculum to an accreditation framework, and a faculty consensus exercise. We discuss challenges to development and implementation of a competency-based framework as well as areas that require further work and development. CONCLUSIONS: The competency-based curriculum is both a new method of learning for residents and, a new method of teaching for faculty. While there are many potential benefits and challenges, this article focuses on the model's utility in terms of flexible learner-centered educational design, as well as its ability to identify learners' strengths and needs.

Validation of a large-scale clinical examination for international medical graduates.

OBJECTIVE: To evaluate a new examination process for international medical graduates (IMGs) to ensure that it is able to reliably assign candidates to 1 of 4 competency levels, and to determine if a global rating scale can accurately stratify examinees into 4 levels of learners: clerks, first-year residents, second-year residents, or practice ready. DESIGN: Validation study evaluating a 12-station objective structured clinical examination. SETTING: Ontario. PARTICIPANTS: A total of 846 IMGs, and an additional 63 randomly selected volunteers from 2 groups: third-year clinical clerks (n = 42) and first-year family medicine residents (n = 21). MAIN OUTCOME MEASURES: The accuracy of the stratification of the examinees into learner levels, the impact of the patient-encounter ratings and postencounter oral questions, and between-group differences in total score. RESULTS: Reliability of the patient-encounter scores, postencounter oral question scores, and the total between-group difference scores was 0.93, 0.88, and 0.76, respectively. Third-year clerks scored the lowest, followed by the IMGs. First-year residents scored highest for all 3 scores. Analysis of variance demonstrated significant between-group differences for all 3 scores (P < .05). Postencounter oral question scores differentiated among all 3 groups. CONCLUSION: Clinical examination scores were capable of differentiating among the 3 groups. As a group, the IMGs seemed to be less competent than the first-year family medicine residents and more competent than the third-year clerks. The scores generated by the postencounter oral questions were the most effective in differentiating between the 2 training levels and among the 3 groups of test takers.