Debriefing Methods for Simulation in Healthcare: A Systematic Review.

ABSTRACT: Debriefing is a critical component in most simulation experiences. With the growing number of debriefing concepts, approaches, and tools, we need to understand how to debrief most effectively because there is little empiric evidence to guide us in their use. This systematic review explores the current literature on debriefing in healthcare simulation education to understand the evidence behind practice and clarify gaps in the literature. The PICO question for this review was defined as "In healthcare providers [P], does the use of one debriefing or feedback intervention [I], compared to a different debriefing or feedback intervention [C], improve educational and clinical outcomes [O] in simulation-based education?" We included 70 studies in our final review and found that our current debriefing strategies, frameworks, and techniques are not based on robust empirical evidence. Based on this, we highlight future research needs.

Application of the R2C2 Model to In-the-Moment Feedback and Coaching.

PURPOSE: The R2C2 (relationship, reaction, content, coaching) model is an iterative, evidence-based, theory-informed approach to feedback and coaching that enables preceptors and learners to build relationships, explore reactions and reflections, confirm content, and coach for change and cocreate an action plan. This study explored application of the R2C2 model for in-the-moment feedback conversations between preceptors and learners and the factors that influence its use. METHOD: A qualitative study using framework analysis through the lens of experiential learning was undertaken with 15 trained preceptor-learner dyads. Data were collected during feedback sessions and follow-up interviews between March 2021 and July 2022. The research team familiarized themselves with the data, used a coding template to document examples of the model's application, reviewed the initial framework and revised the coding template, indexed and summarized the data, created a summary document, examined the transcripts for alignment with each model phase, and identified illustrative quotations and overarching themes. RESULTS: Fifteen dyads were recruited from 8 disciplines (11 preceptors were paired with a single resident [n = 9] or a single medical student [n = 2]; 2 preceptors each had 2 residents). All dyads were able to apply the R2C2 phases of building relationships, exploring reactions and reflections, and confirming content. Many struggled with the coaching components, specifically in creating an action plan and follow-up arrangements. Preceptor skill in applying the model, time available for feedback conversations, and the nature of the relationship impacted how the model was applied. CONCLUSIONS: The R2C2 model can be adapted to contexts where in-the-moment feedback conversations occur shortly after a clinical encounter. Experiential learning approaches applying the R2C2 model are critical. Skillful application of the model requires that learners and preceptors go beyond confirming an area of change and deliberately engage in coaching and cocreating an action plan.

Multisource feedback: an overview of its use and application as a formative assessment.

Multisource feedback (MSF), often termed 360-degree feedback, is a formative performance assessment in which data about an individual's observable workplace behaviors are collected through questionnaires from those interacting with the individual; data are aggregated for anonymity and confidentiality; the aggregated data, along with self-assessment if available, are provided to the individual; and the recipient meets with a trusted individual to review the data and develop an action plan. It is used along the continuum of medical education. This article provides an overview of MSF's utility, its evidence base and cautions.

La rétroaction multisource (RMS), ou rétroaction 360 degrés, est une évaluation formative du rendement dans laquelle des informations sur les comportements observables d'un individu dans son lieu de travail sont recueillies par le biais de questionnaires auprès de ceux avec qui il interagit. Après avoir été agrégées pour garantir l'anonymat et la confidentialité, ces données, et l'auto-évaluation s'il y a lieu, sont remises à la personne évaluée. Accompagnée d'une personne de confiance, elle les examinera et élaborera un plan d'action, qui sera utilisé tout au long du continuum de sa formation médicale. Cet article présente un aperçu de l'utilité et des fondements de la RMS, ainsi que quelques mises en garde.

In-the-Moment Feedback and Coaching: Improving R2C2 for a New Context.

BACKGROUND: The R2C2, a 4-phase feedback and coaching model, builds relationships, explores reactions, determines content and coaches for change, and facilitates formal feedback conversations between clinical supervisors/preceptors and residents. Formal discussions about performance are typically based on collated information from daily encounter sheets, objective structured clinical examinations, multisource feedback, and other data. This model has not been studied in settings where brief feedback and coaching conversations occur immediately after a specific clinical experience. OBJECTIVE: We explored how supervisors adapt the R2C2 model for in-the-moment feedback and coaching and developed a guide for its use in this context. METHODS: Eleven purposefully selected supervisors were interviewed in 2018 to explore where they used the R2C2 model, how they adapted it for in-the-moment conversations, and phrases used corresponding to each phase that could guide design of a new R2C2 in-the-moment model. RESULTS: Participants readily adapted the model to varied feedback situations; each of the 4 phases were relevant for conversations. Phase-specific phrases that could enable effective coaching conversations in a limited amount of time were identified. Data facilitated a revision of the original R2C2 model for in-the-moment feedback and coaching conversations and design of an accompanying trifold brochure to enable its effective use. CONCLUSIONS: The R2C2 in-the-moment model offers a systematic approach to feedback and coaching that builds on the original model, yet addresses time constraints and the need for an iterative conversation between the reaction and content phases. The model enables supervisors to coach and co-create an action plan with residents to improve performance.

Learning Conversations: An Analysis of the Theoretical Roots and Their Manifestations of Feedback and Debriefing in Medical Education.

Feedback and debriefing are experience-informed dialogues upon which experiential models of learning often depend. Efforts to understand each have largely been independent of each other, thus splitting them into potentially problematic and less productive factions. Given their shared purpose of improving future performance, the authors asked whether efforts to understand these dialogues are, for theoretical and pragmatic reasons, best advanced by keeping these concepts unique or whether some unifying conceptual framework could better support educational contributions and advancements in medical education.The authors identified seminal works and foundational concepts to formulate a purposeful review and analysis exploring these dialogues' theoretical roots and their manifestations. They considered conceptual and theoretical details within and across feedback and debriefing literatures and traced developmental paths to discover underlying and foundational conceptual approaches and theoretical similarities and differences.Findings suggest that each of these strategies was derived from distinct theoretical roots, leading to variations in how they have been studied, advanced, and enacted; both now draw on multiple (often similar) educational theories, also positioning themselves as ways of operationalizing similar educational frameworks. Considerable commonality now exists; those studying and advancing feedback and debriefing are leveraging similar cognitive and social theories to refine and structure their approaches. As such, there may be room to merge these educational strategies as learning conversations because of their conceptual and theoretical consistency. Future scholarly work should further delineate the theoretical, educational, and practical relevance of integrating feedback and debriefing.

Transition to practice in radiation oncology: Mind the gap.

INTRODUCTION: Physicians entering independent practice often express apprehension in managing the non-clinical aspects of practice. This study examined the perceived preparedness of radiation oncology (RO) residents for independent practice, identified education gaps, and discussed how these deficiencies could be addressed. MATERIALS AND METHODS: Focus groups with senior RO residents, fellows, new-to-practice radiation oncologists (ROs), and residency program directors were conducted. Data were coded using the Canadian Medical Education Directives for Specialists (CanMEDS) competencies using thematic analysis. RESULTS: Commonly reported gaps in the transition to practice (TtP) for ROs were lack of experience with: practice management, understanding the structure and function of the health care system and how it varies by jurisdiction, financial planning, effective communication and collaboration with other health care team members, creation of accurate and timely documentation, and radiotherapy problem-solving related to treatment planning and evaluation. Suggestions to address these challenges included use of mentorship, educational resources, courses, simulation-based medical education, improved graded responsibility, resident longitudinal clinics, and formal curricula in radiation therapy planning and evaluation. CONCLUSION: There are gaps in TtP education for RO trainees with opportunities for enrichment through the forthcoming implementation of a competency-based medical education framework in 2019. The gap in perceived competency in physician-related radiotherapy tasks may be caused by the complex interaction of clinical workflow processes, people and technology that has led to ineffective integration of trainees. The data are informative to medical education leaders for the development of comprehensive TtP curricula.

Identifying coaching skills to improve feedback use in postgraduate medical education.

OBJECTIVES: Coaching in medical education has recently gained prominence, but minimal attention has been given to key skills and determining how they work to effectively ensure residents are progressing and developing self-assessment skills. This study examined process-oriented and content-oriented coaching skills used in coaching sessions, with particular attention to how supervisors use them to enhance resident acceptance of feedback to enhance learning. METHODS: This qualitative study analysed secondary audiotaped data from 15 supervisors: resident dyads during two feedback sessions, 4 months apart. The R2C2 model was used to engage the resident, build a relationship, explore reactions to feedback, explore resident perceptions of content, and coach for change. Framework analysis was used, including familiarisation with the data, identifying the thematic framework, indexing and charting the data and mapping and interpretation. RESULTS: Process skills included preparation, relationship development, using micro communication skills and techniques to promote reflection and self-assessment by the resident and supervisor flexibility. Content skills related to the specific feedback content included engaging the resident in discussion, ensuring the discussion was collaborative and focused on goal setting, co-developing a Learning Change Plan, ensuring resident commitment and following up on the plan. Together, these skills foster agency in the resident learner. Three overarching themes emerged from the analysis: the interconnectedness of process and content; tensions between encouraging self-direction and ensuring progress and competence; and balancing a coaching dialogue and a teaching monologue. CONCLUSIONS: Effective coaching by supervisors requires a combination of specific process and content skills that are chosen depending on the needs of the individual resident. Mastering these skills helps residents engage and develop agency in their own professional development. These outcomes depend on faculty maintaining a balance between coaching and teaching, encouraging resident self-direction and ensuring progression to competence.

Remediation in Canadian medical residency programs: Established and emerging best practices.

BACKGROUND: Policies to guide remediation in postgraduate medical education exist in all Canadian medical schools. This study examines concordance between these policies and processes, and published "best practices" in remediation. METHOD: We conducted a literature review to identify best practices in the area of remediation. We then reviewed remediation policies from all 13 English medical schools in Canada other than our own and conducted interviews with key informants from each institution. Each policy and interview transcript pair was then reviewed for evidence of pre-defined "best practices." Team members also noted additional potential policy or process enablers of successful remediation. RESULTS: Most policies and processes aligned with some but not all published best practices. For instance, all participating schools tailored remediation strategies to individual resident needs, and a majority encouraged faculty-student relationships during remediation. Conversely, few required the teaching of goal-setting, strategic planning, self-monitoring, and self-awareness. In addition, we identified avoidance of automatic training extension and the use of an educational review board to support the remediation process as enablers for success. DISCUSSION: Remediation policies and practices in Canada align well with published best practices in this area. Based on key informant opinions, flexibility to avoid training extension and use of an educational review board may also support optimal remediation outcomes.

Multisource Feedback and Narrative Comments: Polarity, Specificity, Actionability, and CanMEDS Roles.

INTRODUCTION: Multisource feedback is a questionnaire-based assessment tool that provides physicians with data about workplace behaviors and may combine numeric and narrative (free-text) comments. Little attention has been paid to wording of requests for comments, potentially limiting its utility to support physician performance. This study tested the phrasing of two different sets of questions. METHODS: Two sets of questions were tested with family physicians, medical and surgical specialists, and their medical colleague and coworker respondents. One set asked respondents to identify one thing the participant physician does well and one thing the physician could target for action. Set 2 questions asked what does the physician do well and what might the physician do to enhance practice. Resulting free-text comments provided by respondents were coded for polarity (positive, neutral, or negative), specificity (precision and detail), actionability (ability to use the feedback to direct future activity), and CanMEDS roles (competencies) and analyzed descriptively. RESULTS: Data for 222 physicians (111 physicians per set) were analyzed. A total of 1824 comments (8.2/physician) were submitted, with more comments from coworkers than medical colleagues. Set 1 yielded more comments and were more likely to be positive, semi specific, and very actionable than set 2. However, set 2 generated more very specific comments. Comments covered all CanMEDS roles with more comments for collaborator and leader roles. DISCUSSION: The wording of questions inviting free-text responses influences the volume and nature of the comments provided. Individuals designing multisource feedback tools should carefully consider wording of items soliciting narrative responses.

The R2C2 Model in Residency Education: How Does It Foster Coaching and Promote Feedback Use?

PURPOSE: The authors previously developed and tested a reflective model for facilitating performance feedback for practice improvement, the R2C2 model. It consists of four phases: relationship building, exploring reactions, exploring content, and coaching. This research studied the use and effectiveness of the model across different residency programs and the factors that influenced its effectiveness and use. METHOD: From July 2014-October 2016, case study methodology was used to study R2C2 model use and the influence of context on use within and across five cases. Five residency programs (family medicine, psychiatry, internal medicine, surgery, and anesthesia) from three countries (Canada, the United States, and the Netherlands) were recruited. Data collection included audiotaped site assessment interviews, feedback sessions, and debriefing interviews with residents and supervisors, and completed learning change plans (LCPs). Content, thematic, template, and cross-case analysis were conducted. RESULTS: An average of nine resident-supervisor dyads per site were recruited. The R2C2 feedback model, used with an LCP, was reported to be effective in engaging residents in a reflective, goal-oriented discussion about performance data, supporting coaching, and enabling collaborative development of a change plan. Use varied across cases, influenced by six general factors: supervisor characteristics, resident characteristics, qualities of the resident-supervisor relationship, assessment approaches, program culture and context, and supports provided by the authors. CONCLUSIONS: The R2C2 model was reported to be effective in fostering a productive, reflective feedback conversation focused on resident development and in facilitating collaborative development of a change plan. Factors contributing to successful use were identified.

CPD of the future: a partnership between quality improvement and competency-based education.

CONTEXT: Many of those involved in continuing professional development (CPD) over the past 10 years have engaged in discussions about its goals and activities. Whereas in the past CPD was viewed as an education intervention directed towards the medical expert role, recent research highlights the need to expand the scope of CPD and to promote its more explicit role in improving patient care and health outcomes. Recent developments in quality improvement (QI) and competency-based medical education (CBME), guided by appropriate theories of learning and change, can shed light on how the field might best advance. This paper describes principles of QI and CBME and how they might contribute to CPD, explores theoretical perspectives that inform such an integration and suggests a future model of CPD. DISCUSSION: Continuing professional development seeks to improve patient outcomes by increasing physician knowledge and skills and changing behaviours, whereas QI takes the approach of system and process change. Combining the strengths of a CPD approach with strategies known to be effective from the field of QI has the potential to harmonise the contributions of each, and thereby to lead to better patient outcomes. Similarly, competency-based CPD is envisioned to place health needs and patient outcomes at the centre of a CPD system that will be guided by a set of competencies to enhance the quality of practice and the safety of the health system. CONCLUSIONS: We propose that the future CPD system should adhere to the following principles: it should be grounded in the everyday workplace, integrated into the health care system, oriented to patient outcomes, guided by multiple sources of performance and outcome data, and team-based; it should employ the principles and strategies of QI, and should be taken on as a collective responsibility by physicians, CPD provider organisations, regulators and the health system.

CPD and KT: Models Used and Opportunities for Synergy.

The two fields of continuing professional development (CPD) and knowledge translation (KT) within the health care sector, and their related research have developed as somewhat parallel paths with limited points of overlap or intersection. This is slowly beginning to change. The purpose of this paper is to describe and compare the dominant conceptual models informing each field with the view of increasing understanding and appreciation of the two fields, how they are similar and where they differ, and the current and potential points of intersection. The models include the "knowledge-to-action" (KTA) cycle informing KT, models informing CPD curriculum design and individual self-directed learning, and the Kirkpatrick model for evaluating educational outcomes. When compared through the perspectives of conceptual designs, processes, and outcomes, the models overlap. We also identify shared gaps in both fields (eg, the need to explore the influence of the context in which CPD and KT interventions take place) and suggest opportunities for synergies and for moving forward.

Using the Readiness for Clerkship and Residency Surveys to Evaluate the Effectiveness of Four MD Programs: A Cross-Institutional Generalizability Study.

PURPOSE: The importance of confidence for learning and performance makes learners' perceptions of readiness for the next level of training valuable indicators of curricular success. The "Readiness for Clerkship" (RfC) and "Readiness for Residency" (RfR) surveys have been shown to provide reliable ratings of the relative effectiveness of various aspects of training. This study examines the generalizability of those results. METHOD: Surveys were administered at four medical schools approximately four months after the start of clerkship and eight months after the start of residency during 2013-2015. Collected data were anonymized. A total of 647 medical students and 483 residents participated. RESULTS: Reliabilities of G = 0.8 could be obtained with only 6 to 12 medical students and 8 to 15 residents. Within MD programs, no meaningful differences in item ratings were observed across cohorts. Residents in each school consistently rated themselves higher than clerks on the majority of Medical Expert and Communicator competencies common to both surveys. Similar strengths and weaknesses were identified across programs, but differences were observed on five clerkship items and one residency item. CONCLUSIONS: Across four MD programs, the RfC and RfR surveys provided reliable ratings of the relative effectiveness of aspects of training with small numbers of respondents. The capacity of these surveys to efficiently identify perceived strengths and weaknesses held by cohorts of learners may, thereby, facilitate program improvement.

Responsiveness of a simple tool for assessing change in behavioral intention after continuing professional development activities.

BACKGROUND: Continuing professional development (CPD) activities are one way that new knowledge can be translated into changes in practice. However, few tools are available for evaluating the extent to which these activities change health professionals' behavior. We developed a questionnaire called CPD-Reaction for assessing the impact of CPD activities on health professionals' clinical behavioral intentions. We evaluated its responsiveness to change in behavioral intention and verified its acceptability among stakeholders. METHODS AND FINDINGS: We enrolled 376 health professionals who completed CPD-Reaction before and immediately after attending a CPD activity. We contacted them three months later and asked them to self-report on any behavior change. We compared the mean rankings on each CPD-Reaction construct before and immediately after CPD activities. To estimate its predictive validity, we compared the median behavioral intention score (post-activity) of health professionals reporting a behavior change three months later with the median behavioral intention score of physicians who reported no change. We explored stakeholders' views on CPD-Reaction in semi-structured interviews. Participants were mostly family physicians (62.2%), with an average of 19 years of clinical practice. Post-activity, we observed an increase in intention-related scores for all constructs (P < 0.001) with the most appreciable for the construct beliefs about capabilities. A total of 313 participants agreed to be contacted at follow up, and of these only 69 (22%) reported back. Of these, 43 (62%) self-reported a behavior change. We observed no statistically significant difference in intention between health professionals who later reported a behavior change and those who reported no change (P = 0.30). Overall, CPD stakeholders found the CPD-Reaction questionnaire of interest and suggested potential solutions to perceived barriers to its implementation. CONCLUSION: The CPD-Reaction questionnaire seems responsive to change in behavioral intention. Although CPD stakeholders found it interesting, future implementation will require addressing barriers they identified.

R2C2 in Action: Testing an Evidence-Based Model to Facilitate Feedback and Coaching in Residency.

BACKGROUND: Feedback is increasingly seen as a collaborative conversation between supervisors and learners, where learners are actively and reflectively engaged with feedback and use it to improve. Based on this, and through earlier research, we developed an evidence- and theory-informed, 4-phase model for facilitating feedback and practice improvement-the R2C2 model (relationship, reaction, content, coaching). OBJECTIVE: Our goal was to explore the utility and acceptability of the R2C2 model in residency education, specifically for engaging residents in their feedback and in using it to improve, as well as the factors influencing its use. METHODS: This qualitative study used the principles of design research. We recruited residents and their supervisors in 2 programs, internal medicine and pediatrics. We prepared supervisors to use the R2C2 model during their regular midrotation and/or end-of-rotation feedback sessions with participating residents to discuss their progress and assessment reports. We conducted debriefing interviews with supervisors and residents after each session. We analyzed transcripts as a team using template and content analysis. RESULTS: Of 61 residents, 7 residents (11%) participated with their supervisors (n = 5). Schedules and sensitivity to feedback prevented broader enrollment. Supervisors found the structured R2C2 format useful. Residents and supervisors reported that the coaching phase was novel and helpful, and that the R2C2 model engaged both groups in collaborative, reflective, goal-oriented feedback discussions. CONCLUSIONS: Participants found that using the R2C2 model enabled meaningful feedback conversations, identification of goals for improvement, and development of strategies to meet those goals.

Where is the client/patient voice in interprofessional healthcare team assessments? Findings from a one-day forum.

There is a growing interest in interprofessional care (IPC) as a way to provide better healthcare. However, it is difficult to evaluate this mode of healthcare delivery because identifying the appropriate measurement tool is a challenge, given the wide diversity in team composition and settings. Adding to this complexity is a key gap in the IPC evaluation research: the client/patient perspective. This perspective has generally not been included in the development of IPC healthcare team evaluations. The authors received a Canadian Institute for Health Research Planning Grant to host a one-day forum with 24 participants from across Canada representing health professions such as social work, medicine, occupational therapy, and physical therapy, in addition to researchers, client/patient advocates, and hospital administrators. The overarching goal of the forum was to create a demonstration project that supports the development of an IPC assessment tool for healthcare teams that includes clients/patients. Using a concept mapping methodology, participants discussed client/patient inclusion in IPC assessments, and through a consensus process, chose a demonstration project for further development.

Enhancing the Effectiveness of Significant Event Analysis: Exploring Personal Impact and Applying Systems Thinking in Primary Care.

INTRODUCTION: Significant event analysis (SEA) is well established in many primary care settings but can be poorly implemented. Reasons include the emotional impact on clinicians and limited knowledge of systems thinking in establishing why events happen and formulating improvements. To enhance SEA effectiveness, we developed and tested "guiding tools" based on human factors principles. METHODS: Mixed-methods development of guiding tools (Personal Booklet-to help with emotional demands and apply a human factors analysis at the individual level; Desk Pad-to guide a team-based systems analysis; and a written Report Format) by a multiprofessional "expert" group and testing with Scottish primary care practitioners who submitted completed enhanced SEA reports. Evaluation data were collected through questionnaire, telephone interviews, and thematic analysis of SEA reports. RESULTS: Overall, 149/240 care practitioners tested the guiding tools and submitted completed SEA reports (62.1%). Reported understanding of how to undertake SEA improved postintervention (P < .001), while most agreed that the Personal Booklet was practical (88/123, 71.5%) and relevant to dealing with related emotions (93/123, 75.6%). The Desk Pad tool helped focus the SEA on systems issues (85/123, 69.1%), while most found the Report Format clear (94/123, 76.4%) and would recommend it (88/123, 71.5%). Most SEA reports adopted a systems approach to analyses (125/149, 83.9%), care improvement (74/149, 49.7), or planned actions (42/149, 28.2%). DISCUSSION: Applying human factors principles to SEA potentially enables care teams to gain a systems-based understanding of why things go wrong, which may help with related emotional demands and with more effective learning and improvement.