The impact of deliberate reflection with WISE-MD™ modules on surgical clerkship students' critical thinking: a prospective, randomized controlled pilot study.

PURPOSE: Critical thinking underlies several Association of American Medical Colleges (AAMC)-defined core entrustable professional activities (EPAs). Critical-thinking ability affects health care quality and safety. Tested tools to teach, assess, improve, and nurture good critical-thinking skills are needed. This prospective randomized controlled pilot study evaluated the addition of deliberate reflection (DR), guidance with Web Initiative in Surgical Education (WISE-MD™) modules, to promote surgical clerks' critical-thinking ability. The goal was to promote the application of reflective awareness principles to enhance learning outcomes and critical thinking about the module content. PARTICIPANTS AND METHODS: Surgical clerkship (SC) students were recruited from two different blocks and randomly assigned to a control or intervention group. The intervention group was asked to record responses using a DR guide as they viewed two selected WISE-MD™ modules while the control group was asked to view two modules recording free thought. We hypothesized that the intervention group would show a significantly greater pre- to postintervention increase in critical-thinking ability than students in the control group. RESULTS: Neither group showed a difference in pre- and posttest free-thought critical-thinking outcomes; however, the intervention group verbalized more thoughtful clinical reasoning during the intervention. CONCLUSION: Despite an unsupported hypothesis, this study provides a forum for discussion in medical education. It took a sponsored tool in surgical education (WISE-MD™) and posed the toughest evaluation criteria of an educational intervention; does it affect the way we think? and not just what we learn, but how we learn it? The answer is significant and will require more resources before we arrive at a definitive answer.

Assessing learning in the adaptive curriculum.

BACKGROUND: Medical education is a dynamic process that will continuously evolve to respond to changes in the foundations of medicine, the clinical practice of medicine and in health systems science. PURPOSE: In this paper, we review how assessing learning in such a dynamic environment requires comprehensive flexible and adaptable methodological approaches designed to assess knowledge attainment and transfer, clinical skills/competency development, and ethical/professional behavior. Adaptive assessments should measure the learner's ability to observe where changes in health care delivery are needed and how to implement them. Balancing formative and summative assessments will promote reflective learning so that each student will reach her/his highest potential. From the programmatic perspective, measuring the design and delivery of instruction in relation to students? efforts to achieve competency will improve learning and foster continuous professional development of faculty and advance the science of learning. APPROACH: We describe how two medical schools are approaching adaptive assessment, including using portfolio systems that encompass teaching and learning experiences while offering real-time longitudinal tracking of digital data toward improving learning and provide curricula continuous improvement cycles. Using latest technologies, portfolios produce actionable data displays with precise guidance for learning and program development.

Adaptive medical education research.

This commentary reflects on what the four research articles in this issue of Medical Teacher tell us about adaptive learning and how adaptive medical education research can be conducted. Adaptive medical education researchers must think disruptively and embrace nontraditional collaborations, research methods, and means of dissemination in pursuit of evidence that enhances adaptive learning.

LectureKeepr: A novel approach to studying in the adaptive curriculum.

As medical students search for ways to squeeze more hours of learning into their day, many have turned to technology for quick and efficient ways to study. Most commonly this includes based apps for purchase that involve visual tasks, mental review and repetition. Tasks involve flashcard creation, answering multiple choice questions (MCQs) and schedule planning. They typically require students to visually interact with a computer or smartphone screen. Alternatively, auditory apps may not only offer the benefit of enhanced learning through review and repetition, but also address convenience, portability, and metacognitive development without physical and temporal requirements of visual presentation. Auditory learning is eyes and "hands-free" allowing the learner to contiguously engage in other activities such as physical exercise, traveling, eating, etc. In this randomized, case-comparison study, half of the students were exposed to audio lecture content before class using a mobile application called LectureKeepr (LK) and half were exposed to lecture content before class via written material prepared by the professor. Pre- and post-MCQ quizzes were used to measure differences between groups in knowledge improvement. In addition, student perceptions of the auditory learning experience were assessed by an on-line survey. ANOVA with repeated measures revealed that students in the LK intervention group performed better on the postquiz than the students in the written materials group.

The adaptive medical curriculum: A model for continuous improvement.

This paper describes the medical curriculum designed to foster adaptive expertise. Engaging in the formal and informal curriculum, students learn to achieve desired outcomes in novel situations, perform comfortably with uncertainty, and are often recognized for creative problem-solving. Students learn by asking and answering their own and others' questions. They readily operate at the metacognitive level, anticipating events, self-monitoring, and checking decisions and emotions. A key function of the reflective process is to identify gaps or shortcomings in the thinking process. The adaptive learner shifts into reflective thinking when confronted with complex contextual and situational demands. We are only beginning to understand how to create educational pathways to foster adaptive learning. An essential focus is the adaptive teacher who frames learning and assessment around predictive analytics, reflective spaced practice, and authentic learning material. To be effective, the teacher must engage the learner outside the formal classroom in the parallel curriculum. A major premise is that learning occurs individually and together with peers, teachers, and team members in multiple contexts. During the learning process, the learner readily operates at the metacognitive level, anticipating behavior, self-monitoring and assessing, and checking theirs and others decisions and emotions. The adaptive medical curriculum provides the pathway for such learning.

Medical Student Perceptions of the Learning Environment at the End of the First Year: A 28-Medical School Collaborative.

PURPOSE: Accreditation and professional organizations have recognized the importance of measuring medical students' perceptions of the learning environment, which influences well-being and professional competency development, to optimize professional development. This study was conducted to explore interactions between students' perceptions of the medical school learning environment, student demographic variables, and students' professional attributes of empathy, coping, tolerance of ambiguity, and patient-centeredness to provide ideas for improving the learning environment. METHOD: Twenty-eight medical schools at 38 campuses recruited 4,664 entering medical students to participate in the two-cohort longitudinal study (2010-2014 or 2011-2015). The authors employed chi-square tests and analysis of variance to examine the relationship between Medical School Learning Environment Survey (MSLES) scores and student characteristics. The authors used mixed-effects models with random school and campus effects to test the overall variances accounted for in MSLES scores at the end of the first year of medical school. RESULTS: Student attributes and demographic characteristics differed significantly across schools but accounted for only 2.2% of the total variance in MSLES scores. Medical school campus explained 15.6% of the variance in MSLES scores. CONCLUSIONS: At year's end, students' perceptions toward the learning environment, as reported on the MSLES, differed significantly according to the medical school campus where they trained. Further studies are needed to identify specific factors, such as grading policies, administrative support, and existence of learning communities, which may influence perceptions of the learning environment at various schools. Identifying such variables would assist schools in developing a positive learning environment.

Development of a metacognitive effort construct of empathy during clinical training: a longitudinal study of the factor structure of the Jefferson Scale of Empathy.

Empathy is crucial for effective clinical care but appears to decline during undergraduate medical training. Understanding the nature of this decline is necessary for addressing it. The Jefferson Scale of Empathy (JSE) is used to measure medical students' clinical empathy attitudes. One recent study described a 3-factor model of the JSE. This model was found in responses from matriculating medical students, but little is known about how the factor structure of the scale changes during clinical training. The Learning Environment Study is a longitudinal prospective study of two cohorts from 28 medical schools. At matriculation and at the end of each subsequent year, students self-reported clinical empathy attitudes using the JSE. Data from 4,797 students were randomly partitioned for exploratory (EFA) and confirmatory factor analyses using responses from preclinical and clinical years of medical school. Five models were compared for confirmatory factor analysis: two null models for control, the recent 3-factor model, and the two models resulting from the EFAs of preclinical and clinical year responses. Preclinical year responses yielded a 3-factor model similar to the recent 3-factor model. Clinical year responses yielded a 4-factor model ("feelings," "importance," "ease," and "metacognitive effort") suggesting changes in the structure of clinical empathy attitudes over time. Metacognitive effort showed the largest decline over time. The model is a better fit for both preclinical and clinical responses and may provide more insight into medical students' clinical empathy attitudes than other models. The emergence of metacognitive effort in the clinical years suggests empathy may become more nuanced for students after clinical exposure and may account for much of the observed decline in clinical empathy attitudes.

Expert communication training for providers in community health centers.

BACKGROUND: Community health center (CHC) clinicians describe significant challenges in delivering care for populations with complex health needs. INTERVENTION: A three-workshop series was presented to 102 providers working in CHCs. Training focused on four areas identified through online needs assessment: challenging interactions; patient perspective; health literacy; and motivational interviewing. EVALUATION METHODS: A retrospective pre-post evaluation measured self-perceived change in content knowledge in all four areas. Participants documented commitments to change behaviors across workshops, which were analyzed for recurring themes. RESULTS: Paired t-tests documented improvement in all four content areas. Content analysis of commitments yielded four themes: empowering patients, structuring care, understanding patients, and reflecting purposefully. Of the sixty-eight percent of participants responding to post-workshop queries about their commitments (n=70), 94% report having fully implemented changes in practice behavior or planning to do so. CONCLUSIONS: Providers at CHCs benefit from opportunities to learn and reflect together about communication challenges in practice.

The impact on medical practice of commitments to change following CME lectures: a randomized controlled trial.

BACKGROUND: Self-reported commitment to change (CTC) could be a potentially valuable method to address the need for continuing medical education (CME) to demonstrate clinical outcomes. AIM: This study determines: (1) are clinicians who make CTCs more likely to report changes in their medical practices and (2) do these changes persist over time? METHODS: Intervention participants (N = 80) selected up to three commitments from a predefined list following the lecture, while control participants (N = 64) generated up to three commitments at 7 days post-lecture. At 7 and 30 days post-lecture, participants were queried if any practice change occurred as a result of attending the lecture. RESULTS: About 91% of the intervention group reported practice changes consistent with their commitments at 7 days. Only 32% in the control group reported changes (z = 7.32, p < 0.001). At 30 days, more participants in the intervention group relative to the control group reported change (58% vs. 22%, z = 3.74, p < 0.01). Once a participant from either group made a commitment, there were no differences in reported changes (63% vs. 67%, z = <0.00, p = 0.38). CONCLUSION: Integration of CTC is an effective method of reinforcing learning and measuring outcomes.

Transforming the culture of surgical education: promoting teacher identity through human factors training.

CONTEXT: Promoting a culture of teaching may encourage students to choose a surgical career. Teaching in a human factors (HF) curriculum, the nontechnical skills of surgery, is associated with surgeons' stronger identity as teachers and with clinical students' improved perception of surgery and satisfaction with the clerkship experience. OBJECTIVE: To describe the effects of an HF curriculum on teaching culture in surgery. DESIGN, SETTING, PARTICIPANTS, AND INTERVENTION: Surgeons and educators developed an HF curriculum including communication, teamwork, and work-life balance. MAIN OUTCOME MEASURES: Teacher identity, student interest in a surgical career, student perception of the HF curriculum, and teaching awards. RESULTS: Ninety-two of 123 faculty and residents in a single program (75% of total) completed a survey on teacher identity. Fifteen of the participants were teachers of HF. Teachers of HF scored higher than control participants on the total score for teacher identity (P < .001) and for subcategories of global teacher identity (P = .001), intrinsic satisfaction (P = .001), skills and knowledge (P = .006), belonging to a group of teachers (P < .001), feeling a responsibility to teach (P = .008), receiving rewards (P =.01), and HF (P = .02). Third-year clerks indicated that they were more likely to select surgery as their career after the clerkship and rated the curriculum higher when it was taught by surgeons than when taught by educators. Of the teaching awards presented to surgeons during HF years, 100% of those awarded to attending physicians and 80% of those awarded to residents went to teachers of HF. CONCLUSION: Curricular focus on HF can strengthen teacher identity, improve teacher evaluations, and promote surgery as a career choice.

A human factors curriculum for surgical clerkship students.

HYPOTHESIS: Early introduction of a full-day human factors training experience into the surgical clerkship curriculum will teach effective communication skills and strategies to gain professional satisfaction from a career in surgery. DESIGN: In pilot 1, which took place between July 1, 2007, and December 31, 2008, 50 students received training and 50 did not; all received testing at the end of the rotation for comparison of control vs intervention group performance. In pilot 2, a total of 50 students were trained and received testing before and after rotation to examine individual change over time. SETTING: University of Massachusetts Medical School. PARTICIPANTS: A total of 148 third-year medical students in required 12-week surgical clerkship rotations. INTERVENTIONS: Full-day training with lecture and small-group exercises, cotaught by surgeons and educators, with focus on empathetic communication, time management, and teamwork skills. MAIN OUTCOME MEASURES: Empathetic communication skill, teamwork, and patient safety attitudes and self-reported use of time management strategies. RESULTS: Empathy scores were not higher for trained vs untrained groups in pilot 1 but improved from 2.32 to 3.45 on a 5-point scale (P < .001) in pilot 2. Students also were more likely to ask for the nurse's perspective and to seek agreement on an action plan after team communication training (pilot 1, f = 7.52, P = .007; pilot 2, t = 2.65, P = .01). Results were mixed for work-life balance, with some trained groups scoring significantly lower than untrained groups in pilot 1 and no significant improvement shown in pilot 2. CONCLUSIONS: The significant increase in student-patient communication scores suggests that a brief focused presentation followed by simulation of difficult patient encounters can be successful. A video demonstration can improve interdisciplinary teamwork.

Promoting self-directed learning skills in residency: a case study in program development.

Self-directed learning (SDL) skills are essential for the formation and ongoing competence of today's physicians who work in the context of expanding scientific knowledge and changing health care systems. In 2007-2008, the authors developed a program to promote SDL in the Brown University Family Medicine Residency. Through an iterative process, the project team juggled learning theories (i.e., Knowles' SDL model, Collins' cognitive apprenticeship model, and Quirk's expertise development model) with curricular goals, instructional options, and local constraints to design a practical and theoretically robust intervention.The intervention that emerged from this process features a faculty physician serving as a learning coach who meets individually each month with all second-year residents to assist them in generating learning goals, reflecting on their learning experiences, and practicing evidence-based medicine (EBM) skills. An electronic portfolio serves as a documentation tool that supports reflection; residents record their goals and reflections in the portfolio, which also contains their formative assessments, procedure logs, and special projects. To address the hidden curriculum, the program designers took special care to avoid increasing faculty and resident workload and created a forum for discussion and group reflection. Program evaluation combines qualitative and quantitative methods, such as surveys of and interviews with residents and faculty, to assess changes in residents' SDL and EBM skills and in the program's educational culture. The authors use Kern and colleagues' six-step model for curriculum development to describe both the unfolding of this complex project and the choices that resulted in the current program design.

Human Emotion and Response in Surgery (HEARS): a simulation-based curriculum for communication skills, systems-based practice, and professionalism in surgical residency training.

BACKGROUND: This study examines the development and implementation of a pilot human factors curriculum during a 2-year period. It is one component of a comprehensive 5-year human factors curriculum spanning core competencies of interpersonal and communication skills, systems-based practice, and professionalism and using low-and high-fidelity simulation techniques. STUDY DESIGN: Members of the Department of Surgery and the Center for Clinical Communication and Performance Outcomes jointly constructed a curriculum for PGY1 and PGY2 residents on topics ranging from challenging communication to time and stress management. Video demonstrations, triggers, and simulated scenarios involving acting patients were created by surgeons and medical educators. Pre- and postintervention measures were obtained for communication skills, perceived stress level, and teamwork. Communication skills were evaluated using a series of video vignettes. The validated Perceived Stress Scale and Teamwork and Patient Safety Attitudes survey were used. Residents' perceptions of the program were also measured. RESULTS: Twenty-seven PGY1 residents and 15 PGY2 residents participated during 2 years. Analyses of video vignette tests indicated significant improvement in empathic communication for PGY1 (t = 3.62, p = 0.001) and PGY2 (t = 5.00, p = 0.004). There were no significant changes to teamwork attitudes. Perceived levels of stress became considerably higher. PGY1 residents reported trying 1 to 3 strategies taught in the time management session, with 60% to 75% reporting improvement post-training. CONCLUSIONS: This unique and comprehensive human factors curriculum is shown to be effective in building communication competency for junior-level residents in the human and emotional aspects of surgical training and practice. Continued refinement and ongoing data acquisition and analyses are underway.

The use of simulation in the development of individual cognitive expertise in emergency medicine.

This consensus group from the 2008 Academic Emergency Medicine Consensus Conference, "The Science of Simulation in Healthcare: Defining and Developing Clinical Expertise," held in Washington, DC, May 28, 2008, focused on the use of simulation for the development of individual expertise in emergency medicine (EM). Methodologically sound qualitative and quantitative research will be needed to illuminate, refine, and test hypotheses in this area. The discussion focused around six primary topics: the use of simulation to study the behavior of experts, improving the overall competence of clinicians in the shortest time possible, optimizing teaching strategies within the simulation environment, using simulation to diagnose and remediate performance problems, and transferring learning to the real-world environment. Continued collaboration between academic communities that include medicine, cognitive psychology, and education will be required to answer these questions.

How patients perceive a doctor's caring attitude.

OBJECTIVE: Caring is closely associated with reduced malpractice litigation, adherence to treatment and even symptom relief. Caring also is included in pay for performance formulas as well as widely utilized for quality improvement purposes. Our objective in this prospective qualitative study was to define caring behaviors associated with three challenging encounters: discussing the transition from curative to palliative care, delivering bad news (cancer), and discussing a medical error (misplaced test result). The purpose was to lay the groundwork for the creation of a 'patient-centered' caring attitude checklist that could help the healthcare provider understand and ultimately enhance the patient's experience of care. METHODS: Groups of randomly selected lay people, henceforth referred to as patients: (1) engaged in 'think aloud' exercises to help create a 15-item caring behavior checklist; (2) used the checklist to rate videotapes of simulated challenging encounters conducted by twenty primary care physicians (total of 600 ratings sets); and (3) participated in 12 separate 1.5 h focus groups discussing the caring (and non-caring) behaviors exhibited in videotapes of the highest and lowest rated encounters. RESULTS: Thirteen behaviors emerged as focal for describing a doctor's caring attitude but with disagreement as to whether specific examples of these behaviors were 'caring' or 'uncaring.' For example, although the concept of empathic inquiry was considered important by most patients, the physician question, "Is there someone you can call or talk with" (about a cancer diagnosis) was interpreted by one patient as 'very caring' while another was 'impressed with how uncaring' the statement appeared. CONCLUSION: At the conceptual level there is a set of behaviors that represent caring, however, the manifestation of these behaviors is 'in the eye of the beholder.' The most important element of caring may not be the set of behaviors but a set of underlying abilities that include taking the patient's perspective and reflecting on the patient's responses. PRACTICE IMPLICATIONS: Medical education must focus on the underlying abilities of caring.

Applying patient perspectives on caring to curriculum development.

OBJECTIVE: To report how patient viewpoints on caring inform curriculum development for teaching sessions on delivering bad news, making the transition to palliative care, and communicating about a medical error. METHODS: We conducted focus groups that used patients recruited from the surrounding community to view videotapes of physicians delivering bad news, talking with a patient about palliative care, and communicating about a medical error. We used focus group results, combined with evidence from the medical literature to inform curriculum content for workshops conducted with Internal Medicine and Family Medicine residents at the University of Massachusetts Medical School. RESULTS: Patient perspectives on caring gathered through focus groups differed in significant ways from the existing medical literature on caring when providers are communicating in the challenging situations that were depicted. Our data pointed out that individual reactions were unique and sometimes contradictory in that one person saw behavior as caring that others thought was uncaring. Participants often used qualifiers in their comments like "appropriate" amounts of information, "measured" empathy, chooses words carefully to reflect the relative nature of caring. "Arranges to meet healthcare needs," an issue that extends beyond the encounter, was seen as a new component of caring not previously described. Applying these concepts to curriculum required that we not only focus on the behavioral skills involved in these tasks, but also the processes of assessing patient's informational and emotional needs, and then taking steps to meet them, while adjusting behavior in real time to meet patients needs for caring. Workshops delivered were highly evaluated by residents. CONCLUSION: Patient perspectives on caring when providers deliver bad news, discuss transitions to palliative care, and communicate about a medical error reinforce that patient expectations for caring are highly contextualized and physician behavior needs to be individualized. We taught residents not only behavioral skills, but also the process skills of anticipating patient reactions, recognizing patient clues, planning and choosing effective strategies on the fly, and assessing one's own performance characteristic of communication expertise. PRACTICE IMPLICATIONS: Teaching caring attitudes with challenging communication tasks requires that learners appreciate and value not only caring behaviors but also learn the process by which they must adjust and titrate their actions to meet patient needs.

The video-based test of communication skills: description, development, and preliminary findings.

BACKGROUND: The importance of assessing physician-patient communication skills is widely recognized, but assessment methods are limited. Objective structured clinical examinations are time-consuming and resource intensive. For practicing physicians, patient surveys may be useful, but these also require substantial resources. Clearly, it would be advantageous to develop alternative or supplemental methods for assessing communication skills of medical students, residents, and physicians. DESCRIPTION: The Video-based Test of Communication Skills (VTCS) is an innovative, computer-administered test, consisting of 20 very short video vignettes. In each vignette, a patient makes a statement or asks a question. The examinee responds verbally, as if it was a real encounter and he or she were the physician. Responses are recorded for later scoring. Test administration takes approximately 1 h. EVALUATION: Generalizability studies were conducted, and scores for two groups of physicians predicted to differ in their communication skills were compared. Preliminary results are encouraging; the estimated g coefficient for the communication score for 20-vignette test (scored by five raters) is 0.79; g for the personal/affective score under the same conditions is 0.62. Differences between physicians were in the predicted direction, with physicians considered "at risk" for communication difficulties scoring lower than those not so identified. CONCLUSIONS: The VTCS is a short, portable test of communication skills. Results reported here suggest that scores reflect differences in skill levels and are generalizable. However, these findings are based on very small sample sizes and must be considered preliminary. Additional work is required before it will be possible to argue confidently that this test in particular, and this approach to testing communication skills in general, is valuable and likely to make a substantial contribution to assessment in medical education.

The Family Medicine Curriculum Resource Project: implications for faculty development.

Faculty development implications related to implementing the Family Medicine Curriculum Resource (FMCR) Project provide an opportunity to look at the recommendations of the Society of Teachers of Family Medicine's federally funded Faculty Futures Initiative (FFI) and the recent Future of Family Medicine (FFM) project. Implications for faculty development include the importance of the clerkship setting, originally defined in 1991, with new features added in today's practice environment as outlined by the FFM and the changing assumptions in approaching faculty development. Previously, faculty development focused on teaching learners to master current knowledge. Now, faculty must teach learners how to master new competencies throughout their lives; learners need to learn how they and others learn now. Teaching must focus on how to learn in the future as well as what to learn for the present. Competence ("what individuals know or are able to do in terms of knowledge, skills, and attitudes") has become the focus of curriculum development efforts over the last few years and most appropriately serves as the focus of curriculum development in the FMCR Project. Implications for developing teachers and preceptors focus on the skills and circumstances required to teach and evaluate all types (cognitive, metacognitive, and affective) of competence. In the new culture, novel teaching methods will serve as the focus of faculty development in teaching and of educational ("best practices") research.

Initial testing of an instrument to measure teacher identity in physicians.

BACKGROUND: A previous study described 7 elements of teacher identity: intrinsic satisfaction from teaching, knowledge and skill about teaching, belonging to a community of teachers, receiving rewards for teaching, believing that being a doctor means being a teacher, feeling a responsibility to teach, and sharing clinical expertise. PURPOSE: To conduct the initial testing of an instrument to measure the 7 elements of teacher identity in clinical educators and to consider the potential applications of such an instrument. METHODS: A 37-item questionnaire was mailed to 153 preceptors of preclinical students. Categories reflected the elements of teacher identity listed here. Demographic data were collected. Means, alphas, ANOVAs, and paired t tests were calculated. RESULTS: Of 153 preceptors, 127 (83%) completed the questionnaire. Cronbach's alpha for the overall scale and several subscales were high. Salaried physicians and those who had completed a faculty development program scored significantly higher on several subscales than physicians who volunteered to teach or who did not have faculty development. CONCLUSIONS: This study provides preliminary evidence that teacher identity can be measured and that preceptors do not respond as a homogeneous group. Assessing teacher identity may be helpful to medical schools looking to identify and support physicians who teach.