Assessing commitment to reflection: perceptions of medical students.

Background: While developing reflection skills is considered important by educators, the assessment of these skills is often associated with unintended negative consequences. In the context of a mandatory longitudinal course that aims to promote the development of reflection on professional identity, we assessed students' commitment to reflection. This study explores students' perception of this assessment by their mentor. Methods: We conducted a qualitative descriptive study using semi-structured interviews with twenty-one 1st and six 2nd year medical students. Thematic analysis was informed by Braun and Clarke's six-step approach. Results: We identified four main themes: 1- assessment as a motivator, 2- consequences on authenticity, 3- perception of inherent subjectivity, and 4 - relationship with the mentor. Conclusions: In the context of assessing reflection skills in future physicians, we observed that students -when assessed on the process of reflection- experienced high motivation but were ambivalent on the question of authenticity. The subjectivity of the assessment as well as the relationship with their mentor also raises questions. Nevertheless, this assessment approach for reflective skills appears to be promising in terms of limiting the negative consequences of assessment.

Contexte: Malgré l'importance que les éducateurs attribuent à l'acquisition de compétences de réflexion, l'évaluation de ces compétences entraîne souvent des conséquences négatives involontaires. Dans le cadre d'un cours longitudinal obligatoire visant à promouvoir le développement de la réflexion sur l'identité professionnelle, nous avons évalué l'engagement des étudiants à cultiver leurs compétences de réflexion. Cette étude explore leur perception de cette évaluation menée par leur mentor. Méthodes: Nous avons réalisé une étude qualitative descriptive à l'aide d'entretiens semi-structurés avec vingt-et-un étudiants en médecine de première année et six étudiants en médecine de deuxième année. Notre analyse thématique repose sur l'approche en six étapes de Braun et Clarke. Résultats: Nous avons identifié quatre thèmes principaux : 1 ­ l'évaluation comme facteur de motivation, 2 ­ les conséquences sur l'authenticité, 3 ­ la perception de la subjectivité inhérente, et 4 ­ la relation avec le mentor. Conclusions: Dans le contexte de l'évaluation des compétences de réflexion des futurs médecins, focalisée plus particulièrement sur le processus de réflexion, les étudiants se sont montrés très motivés, mais incertains quant à son authenticité. La subjectivité de l'évaluation et la relation avec leur mentor soulèvent également des interrogations. Néanmoins, cette approche d'évaluation des compétences réflexives semble prometteuse dans la mesure où elle permet de limiter les conséquences négatives de l'évaluation.

Quality of Narratives in Assessment: Piloting a List of Evidence-Based Quality Indicators.

Background & Need for Innovation: Appraising the quality of narratives used in assessment is challenging for educators and administrators. Although some quality indicators for writing narratives exist in the literature, they remain context specific and not always sufficiently operational to be easily used. Creating a tool that gathers applicable quality indicators and ensuring its standardized use would equip assessors to appraise the quality of narratives. Steps taken for Development and Implementation of innovation: We used DeVellis' framework to develop a checklist of evidence-informed indicators for quality narratives. Two team members independently piloted the checklist using four series of narratives coming from three different sources. After each series, team members documented their agreement and achieved a consensus. We calculated frequencies of occurrence for each quality indicator as well as the interrater agreement to assess the standardized application of the checklist. Outcomes of Innovation: We identified seven quality indicators and applied them on narratives. Frequencies of quality indicators ranged from 0% to 100%. Interrater agreement ranged from 88.7% to 100% for the four series. Critical Reflection: Although we were able to achieve a standardized application of a list of quality indicators for narratives used in health sciences education, it does not exclude the fact that users would need training to be able to write good quality narratives. We also noted that some quality indicators were less frequent than others and we suggested a few reflections on this.

Narrative Assessments in Higher Education: A Scoping Review to Identify Evidence-Based Quality Indicators.

PURPOSE: Narrative comments are increasingly used in assessment to document trainees' performance and to make important decisions about academic progress. However, little is known about how to document the quality of narrative comments, since traditional psychometric analysis cannot be applied. The authors aimed to generate a list of quality indicators for narrative comments, to identify recommendations for writing high-quality narrative comments, and to document factors that influence the quality of narrative comments used in assessments in higher education. METHOD: The authors conducted a scoping review according to Arksey & O'Malley's framework. The search strategy yielded 690 articles from 6 databases. Team members screened abstracts for inclusion and exclusion, then extracted numerical and qualitative data based on predetermined categories. Numerical data were used for descriptive analysis. The authors completed the thematic analysis of qualitative data with iterative discussions until they achieved consensus for the interpretation of the results. RESULTS: After the full-text review of 213 selected articles, 47 were included. Through the thematic analysis, the authors identified 7 quality indicators, 12 recommendations for writing quality narratives, and 3 factors that influence the quality of narrative comments used in assessment. The 7 quality indicators are (1) describes performance with a focus on particular elements (attitudes, knowledge, skills); (2) provides a balanced message between positive elements and elements needing improvement; (3) provides recommendations to learners on how to improve their performance; (4) compares the observed performance with an expected standard of performance; (5) provides justification for the mark/score given; (6) uses language that is clear and easily understood; and (7) uses a nonjudgmental style. CONCLUSIONS: Assessors can use these quality indicators and recommendations to write high-quality narrative comments, thus reinforcing the appropriate documentation of trainees' performance, facilitating solid decision making about trainees' progression, and enhancing the impact of narrative feedback for both learners and programs.

Reflecting on professional identity in undergraduate medical education: implementation of a novel longitudinal course.

BACKGROUND: Today's healthcare professionals face numerous challenges. Improving reflection skills has the potential to contribute to the better management of complex patients and healthcare systems, as well as to improve professional practice. However, the question of how reflection skills can inform professional identity development at the undergraduate medical education level remains unanswered. APPROACH: The authors developed and implemented a 4-year course that aims to engage students in a reflective process to increase their awareness of their professional identity development. The course is structured around three types of pedagogical activities: workshops, reflections deposited in an electronic portfolio, and individual discussions with mentors. EVALUATION: Sixty-four 1st year students (33%) and 17 mentors (50%) from the 2017-2018 cohort completed evaluation questionnaires. For the 2018-2019 academic year, 73 1st year students (34%) and 27 2nd year students (14%), as well as 20 1st year (59%) and 19 2nd year mentors (57%) replied. Students and mentors considered that the pedagogical activities contributed to the development of students' professional identity through the acquisition of reflection skills, but some elements were perceived as challenging, notably, completing the portfolio, finding a subject to reflect about and the timing of the proposed activities. REFLECTION: An important preoccupation when wanting to foster the development of professional identity through the acquisition of reflection skills is the authenticity of students' reflection. We tried to favor authentic reflection, by having a mentee-mentor pair throughout the entire 4­year course. A rigorous evaluation process helped us identify and promptly correct issues as they surfaced.