Stages of Milestones Implementation: A Template Analysis of 16 Programs Across 4 Specialties.

BACKGROUND: Since 2013, US residency programs have used the competency-based framework of the Milestones to report resident progress and to provide feedback to residents. The implementation of Milestones-based assessments, clinical competency committee (CCC) meetings, and processes for providing feedback varies among programs and warrants systematic examination across specialties. OBJECTIVE: We sought to determine how varying assessment, CCC, and feedback implementation strategies result in different outcomes in resource expenditure and stakeholder engagement, and to explore the contextual forces that moderate these outcomes. METHODS: From 2017 to 2018, interviews were conducted of program directors, CCC chairs, and residents in emergency medicine (EM), internal medicine (IM), pediatrics, and family medicine (FM), querying their experiences with Milestone processes in their respective programs. Interview transcripts were coded using template analysis, with the initial template derived from previous research. The research team conducted iterative consensus meetings to ensure that the evolving template accurately represented phenomena described by interviewees. RESULTS: Forty-four individuals were interviewed across 16 programs (5 EM, 4 IM, 5 pediatrics, 3 FM). We identified 3 stages of Milestone-process implementation, including a resource-intensive early stage, an increasingly efficient transition stage, and a final stage for fine-tuning. CONCLUSIONS: Residency program leaders can use these findings to place their programs along an implementation continuum and gain an understanding of the strategies that have enabled their peers to progress to improved efficiency and increased resident and faculty engagement.

The Role of Gender in the Experience and Impact of Recertification Exam Preparation: A Qualitative Study of Primary Care Physicians.

Background: Little is known about how physicians experience preparing for board recertification examinations. As women make up a growing proportion of the primary care physician workforce, we aimed to explore how primary care physicians experience the personal and professional impacts of recertification examination preparation activities, and whether these impacts differ by gender. Materials and Methods: We conducted exploratory qualitative semistructured interviews with 80 primary care physicians, who had recently taken either the American Board of Family Medicine or American Board of Internal Medicine recertification examination and who practice outpatient care. We used an iterative recruitment approach to obtain a representative sample. We applied a team-based constant comparative analytic approach to identify and categorize themes related to how preparing for the recertification examination impacted their personal or professional lives, and then compared these themes by physician gender. Results: We interviewed 41 male and 39 female participants. Physicians most frequently described taking time from personal rather than professional activities to study, but often said this was "no big deal." Physicians described impacts on personal life such as missing out on family or leisure time, conflicts with parenting responsibilities, and an increased reliance on their spouse for domestic and childcare duties. Female physicians more frequently described parenting and leisure time impacts than males did. Conclusions: Recertification examination preparation impacts physicians' personal lives in a variety of ways and are sometimes experienced differently along gendered lines. These findings suggest opportunities for employers, payers, and specialty boards to help physicians ease potential burdens related to maintaining board certification.

How Physicians Prepare for Maintenance of Certification Exams: A Qualitative Study.

PURPOSE: Little is known about how board-certified physicians prepare for their periodic maintenance of certification (MOC) examinations. This qualitative study explores how physicians experience MOC exam preparation: how they prepare for the exams and decide what to study and how exam preparation compares with what they normally do to keep their medical knowledge current. METHOD: Between September 2016 and March 2017, the authors interviewed 80 primary care physicians who had recently taken either the American Board of Family Medicine or American Board of Internal Medicine MOC exam. They analyzed transcripts and notes from these interviews looking for patterns and emergent themes, using the constant comparative method and a social practice theory perspective. RESULTS: Most interviewees studied for their MOC exams by varying from their routines for staying current with medical knowledge, both by engaging with a different scope of information and by adopting different study methods. Physicians described exam preparation as returning to a student/testing mindset, which some welcomed and others experienced negatively or with ambivalence. CONCLUSIONS: What physicians choose to study bounds what they can learn from the MOC exam process and, therefore, also bounds potential improvements to their patient care. Knowing how physicians actually prepare, and how these activities compare with what they do when not preparing for an exam, can inform debates over the value of requiring such exams, as well as conversations about how certification boards and other key stakeholders in physicians' continuing professional development could improve the MOC process.

Tiny Moments Matter: Promoting Professionalism in Everyday Practice.

INTRODUCTION: Professionalism rests upon a number of individual, environmental, and societal level factors, leading to specific professional behavior in specific situations. Focusing on professional lapses to identify and remediate unprofessional physicians is incomplete. We explored professionalism in practicing internal medicine physicians in the context of everyday practice, to highlight how typical experiences contribute to positive, yet often unnoticed, professional behavior. METHODS: In-depth interviews were used to uncover 13 physicians' most meaningful experiences of professionalism. Data were collected and analyzed using a grounded theory approach. RESULTS: Results revealed several themes around which physicians embody professionalism in their daily lives. Physicians feel most professional when they are able to connect and establish trust with patients and colleagues and when they serve as positive role models to others. Physicians conceptualize professionalism as a dynamic and evolving competency, one that requires a lifelong commitment and that provides opportunities for lifelong learning. DISCUSSION: Focusing on actual perceptions of experiences in practice offers important insights into how physicians think about professionalism beyond a traditional remediation and lapses perspective. Physicians often go out of their way to connect with patients and colleagues, serving and modeling for others, often at the expense of their own work-life balance. These moments help to infuse energy and positivity into physician practices during a time when physicians may feel overburdened, overscheduled, and overregulated. Understanding professionalism as developmental helps frame professionalism as a lifelong competency subject to growth and modification over time.

US internists' awareness and use of overtreatment guidelines: a national survey.

OBJECTIVES: To assess physician views and perceived adoption of overtreatment guidelines and measure whether adoption of these guidelines influenced the recommendation of a targeted service. STUDY DESIGN: A cross-sectional survey mailed from July 2014 to January 2015 to 902 internists who completed residency between 2003 and 2013, randomly selected from the American Medical Association Masterfile. METHODS: Poisson regression was used to model the rate of recommending a targeted service included in the guidelines, based on the level of guideline adoption. RESULTS: A total of 456 physicians responded (51% response rate). Most expressed familiarity with overtreatment guidelines (88.5%), a comfort level with discussing these guidelines with patients (79.9%), and described overtreatment guidelines as a useful tool in their practice (81.6%). Physicians in the highest tertile of guideline adoption reported double-digit rates of recommending antibiotics for sinusitis (29.7%), mammogram at end of life (16.5%), and electrocardiogram testing for asymptomatic patients (11.0%). Physicians in the bottom tertile of guideline adoption reported lower rates of recommending x rays (-12.0%; 95% confidence interval [CI], -19.4% to -4.5%; P = .002), magnetic resonance imaging for lower back pain (-4.8%; 95% CI, -8.1% to -1.5%; P = .004), and cardiac testing for asymptomatic patients (-10.2%; 95% CI, -18.9% to -1.5%; P = .02). CONCLUSIONS: US internal medicine physicians who completed residency between 2003 and 2013 reported high levels of adoption of overtreatment guidelines. Physicians who reported the highest levels of guideline adoption reported recommending services targeted by these guidelines in their practice.

How hospitalists work to pull healthcare teams together.

PURPOSE: The purpose of this paper is to document everyday practices by which hospitalist physicians negotiate barriers to effective teamwork. DESIGN/METHODOLOGY/APPROACH: Ethnographic observation with a sample of hospitalists chosen to represent a range of hospital and practice types. FINDINGS: Hospitals rely on effective, interprofessional teamwork but typically do not support it. Hospitalist physicians must bridge the internal boundaries within their hospitals to coordinate their patients' care, but they face challenges - scattered patients, fragmented information, uncoordinated teams, and unreliable processes - that can impact the timeliness and safety of care. Hospitalists largely rely on personal presence and memory to deal with these challenges. Some invent low-tech supports for teamwork, but these are typically neither tested nor shared with others. Formal support for teamwork, primarily case management rounds, is applied unevenly and may not be respected by all team members. RESEARCH LIMITATIONS/IMPLICATIONS: The findings are drawn from observation over a limited period of time with a small, purposefully chosen sample of physicians and hospitals. Practical implications - Hospitals must recognize the issues hospitalists and other providers face, evaluate and disseminate supports for teamwork, and make interprofessional teamwork a core feature of hospital design and evaluation. ORIGINALITY/VALUE: The authors show the nuances of how hospitalists struggle to practice teamwork in a challenging context, and how the approaches they take (relying on memory and personal presence) do not address, and may actually contribute to, the system-level problems they face.

How faculty members experience workplace-based assessment rater training: a qualitative study.

CONTEXT: Direct observation of clinical skills is a common approach in workplace-based assessment (WBA). Despite widespread use of the mini-clinical evaluation exercise (mini-CEX), faculty development efforts are typically required to improve assessment quality. Little consensus exists regarding the most effective training methods, and few studies explore faculty members' reactions to rater training. OBJECTIVES: This study was conducted to qualitatively explore the experiences of faculty staff with two rater training approaches - performance dimension training (PDT) and a modified approach to frame of reference training (FoRT) - to elucidate how such faculty development can be optimally designed. METHODS: In a qualitative study of a multifaceted intervention using complex intervention principles, 45 out-patient resident faculty preceptors from 26 US internal medicine residency programmes participated in a rater training faculty development programme. All participants were interviewed individually and in focus groups during and after the programme to elicit how the training influenced their approach to assessment. A constructivist grounded theory approach was used to analyse the data. RESULTS: Many participants perceived that rater training positively influenced their approach to direct observation and feedback, their ability to use entrustment as the standard for assessment, and their own clinical skills. However, barriers to implementation and change included: (i) a preference for holistic assessment over frameworks; (ii) challenges in defining competence; (iii) difficulty in changing one's approach to assessment, and (iv) concerns about institutional culture and buy-in. CONCLUSIONS: Rater training using PDT and a modified approach to FoRT can provide faculty staff with assessment skills that are congruent with principles of criterion-referenced assessment and entrustment, and foundational principles of competency-based education, while providing them with opportunities to reflect on their own clinical skills. However, multiple challenges to incorporating new forms of training exist. Ongoing efforts to improve WBA are needed to address institutional and cultural contexts, and systems of care delivery.

Assessing interprofessional teamwork: pilot test of a new assessment module for practicing physicians.

INTRODUCTION: Teamwork is a basic component of all health care, and substantial research links the quality of teamwork to safety and quality of care. The TEAM (Teamwork Effectiveness Assessment Module) is a new Web-based teamwork assessment module for practicing hospital physicians. The module combines self-assessment, multisource feedback from members of other professions and specialties with whom the physician exercises teamwork, and a structured review of those data with a peer to develop an improvement plan. METHODS: We conducted a pilot test of this module with hospitalist physicians to evaluate the feasibility and usefulness of the module in practice, focusing on these specific questions: Would physicians in hospitals of different types and sizes be able to use the module; would the providers identified as raters respond to the request for feedback; would the physicians be able to identify one or more "trusted peers" to help analyze the feedback; and how would physicians experience the module process overall? RESULTS: 20 of 25 physicians who initially volunteered for the pilot completed all steps of the TEAM, including identifying interprofessional teammates, soliciting feedback from their team, and identifying a peer to help review data. Module users described the feedback they received as helpful and actionable, and indicated this was information they would not have otherwise received. CONCLUSIONS: The results suggest that a module combining self-assessment, multisource feedback, and a guided process for interpreting these data can provide help practicing hospital physicians to understand and potentially improve their interprofessional teamwork skills and behaviors.

Professionalism and maintenance of certification: using vignettes describing interpersonal dilemmas to stimulate reflection and learning.

INTRODUCTION: Physicians do not always agree on "rules" governing behavior in professionally challenging situations. Little is known about contextual factors that influence physician approaches in these situations. We explored the individual-, social-, and societal-level factors that physicians consider when responding to 2 common professional dilemmas. We were particularly interested in knowing the extent to which physicians engage in self-reflection as a result of responding to the vignettes. METHODS: A cross-sectional Web-based survey was sent to a random sample of 396 physicians, prompting them to respond to scripted professional dilemmas. RESULTS: A total of 120 physicians responded, yielding a response rate of 32.6%. Physicians responded to these dilemmas in highly variable ways, negotiating a complex array of contextual factors in their decisions. Interacting factors involving individual-level physician (eg, worry, guilt), patient (eg, nature of medical condition or relationship with patient), and social/societal (eg, policy, what peers or colleagues do) were important drivers in physician responses. Qualitative analysis revealed that several interacting themes guide physician approaches to professional dilemmas: patient welfare; types of patients; political, ethical, or legal issues; guiding principles; values; rules; and habits. DISCUSSION: Physicians negotiate a complex set of individual-, social-, and societal-level factors in response to professional dilemmas. This finding has important implications for the promotion and assessment of professional behavior in practicing physicians. Reflecting on scenarios may be an opportunity for physicians to learn about how and why they make decisions in difficult situations.

Doing what might be "wrong": understanding internists' responses to professional challenges.

PURPOSE: To develop a deeper understanding of the complexity of physicians' decision making when faced with professional challenges. METHOD: Using constructivist grounded theory, the authors conducted a secondary analysis of transcripts from focus groups with 40 internists in 2011. Participants responded to scripted professional challenge scenarios. The authors analyzed the transcripts for instances in which participants discussed "doing what might be wrong" (i.e., something that goes against their values or others' expectations). They used the theory of planned behavior (TPB), which posits that intention to act is predicted by attitudes, subjective norms, and perceived behavioral control, to understand the findings in a broader context. RESULTS: The theme of "doing what might be wrong" was pervasive, particularly in response to scenarios involving stewardship, nonpatients' requests for advice or care, or requests for e-mail access. Participants' rationales for suggested behaviors included a desire to keep patients happy and be (or appear) helpful. Modifiers of those responses included type of patient, physician's relationship with the patient, and comfort level with the request. Consistent with the TPB, attitudes or beliefs about the intended behavior, subjective norms, and perceived behavioral control influenced decision making. CONCLUSIONS: Physicians often do what might be wrong when they are asked to do something that goes against their values and beliefs, by patients, others, or as perceived by their organizations. Actions are often rationalized as being done for the right reasons. These findings should inform the development of educational initiatives to support physicians in acting in accordance with their ideals.

The 'special obligations' of the modern Hippocratic Oath for 21st century medicine.

CONTEXT: Profound advances and discoveries in medicine have markedly improved the lives of many over the 50 years since the modern Hippocratic Oath was written. Regrettably, these advances were and continue to be implemented suboptimally and inequitably across the globe. 'Special obligations to all my fellow humans' is an important theme of the modern Oath. From this perspective, we reflect on the special obligations of the medical profession, and examine how these obligations have changed over the past 50 years. METHODS: We draw from perspectives of the social contract, professionalism, quality improvement, patient safety and a group of 31 international colleagues involved in medical education as we examine these obligations for individual doctors, health care institutions and medical education systems. The perspectives of the 31 clinician-educators helped us to situate the meaning of the theme of 'special obligations' in the context of challenges facing medical education and health care in the 21st century. OBSERVATIONS: Improving the quality of care and patient safety, and reducing health care disparities are now paramount as 'special obligations' for doctors, health care systems and medical education organisations, and require us to work collectively and collaboratively in an increasingly interconnected world. In our view, traditions such as the Hippocratic Oath will be worthy of public support only when the medical profession demonstrates in meaningful and transparent ways that it is meeting its social and civic obligations to make the world, not just health care, a better place.

Patients, providers, and systems need to acquire a specific set of competencies to achieve truly patient-centered care.

Studies show that patients want to be more involved in their own health care. Yet insufficient attention has been paid to the specific competencies of both patients and providers that are needed to optimize such patient engagement and shared decision making. In this article we address the knowledge, skills, and attitudes that patients, physicians, and health care systems require to effectively engage patients in their health care. For example, many patient-physician interactions still follow the traditional office visit format, in which the patient is passive, trusting, and compliant. We recommend imaginative models for redesigned office care, restructured reimbursement schemes, and increased support services for patients and professionals. We present three clinical scenarios to illustrate how these competencies must work together. We conclude that effective shared decision making takes time to deliver proficiently and that among other measures, policy makers must change payment models to focus on value and support education and discussion of competencies for a modern health care system.

Optimizing the implementation of practice improvement modules in training: lessons from educators.

BACKGROUND: The American Board of Internal Medicine approved the use of Practice Improvement Modules (PIMs) to help training programs teach and assess practice-based learning and improvement (PBLI) and systems-based practice (SBP). METHODS: We surveyed individuals who ordered a PIM in a residency or fellowship training program between June 2006 and August 2009. The 43 programs that volunteered to participate completed a 30-minute anonymous online survey. RESULTS: Program directors or associate program directors led the PIM process in 30 programs (70%). Trainees' degrees of involvement in PIMs were highly variable between programs, and several respondents felt that trainees were either not sufficiently engaged or not engaged with enough consistency. The most common activity for trainee involvement was data collection through patient surveys or chart review, although only 17 programs (40%) provided protected time for this activity. Few trainees participated in higher level activities such as data analysis or identification for areas of improvement or were given leadership roles; yet most respondents reported that completing the PIM helped trainees learn basic principles of QI and develop competence in PBLI and SBP and that PIM completion improved the program's ability to develop QI initiatives and resulted in program or institutional improvements, including sustainable improvement in patient care. Most respondents reported that the outcome warranted the effort to complete PIMs. CONCLUSIONS: PIMs may be a valuable but underused educational experience for trainees as well as training programs. Focusing on particular themes and facets of PIMs may facilitate implementation.

A new tool to give hospitalists feedback to improve interprofessional teamwork and advance patient care.

Teamwork is a vital skill for health care professionals, but the fragmented systems within which they work frequently do not recognize or support good teamwork. The American Board of Internal Medicine has developed and is testing the Teamwork Effectiveness Assessment Module (TEAM), a tool for physicians to evaluate how they perform as part of an interprofessional patient care team. The assessment provides hospitalist physicians with feedback data drawn from their own work of caring for patients, in a way that is intended to support immediate, concrete change efforts to improve the quality of patient care. Our approach demonstrates the value of looking at teamwork in the real world of health care-that is, as it occurs in the actual contexts in which providers work together to care for patients. The assessment of individual physicians' teamwork competencies may play a role in the larger effort to bring disparate health professions together in a system that supports and rewards a team approach in hope of improving patient care.

"It depends": results of a qualitative study investigating how practicing internists approach professional dilemmas.

PURPOSE: Context has a critical influence on individuals' behaviors and is essential to understanding lapses in professionalism, yet little is known about contextual factors relevant to practicing physicians. This study used standardized professionalism dilemmas, or challenges, to explore practicing internists' reasoning in their handling of typical challenges. METHOD: In spring 2011, the authors created several professional challenges relevant to physicians in practice and conducted five focus groups with practicing internists (n = 40). Each group discussed five or six of the challenges, and the facilitators specifically asked what the participants would do and why. The authors used constructivist grounded theory to analyze the transcripts. RESULTS: The scenarios were effective in eliciting discussion and debate. Analysis revealed many guiding principles (e.g., patient welfare, keeping patients happy) that influenced physicians in their approach to professionalism challenges, but these principles were highly context-dependent. The authors found individuals' responses to be malleable and subject to much modification depending on input from peers. Responses often shifted in an iterative and complex manner, depending on factors such as the "type" of patients (including the physician's personal feelings toward them), the nature of the illness or diagnosis, and the physician's relationships with others. CONCLUSIONS: Despite recognizing and articulating basic guiding principles of professionalism, physicians' approaches to professional challenges were subject to multiple, interdependent, idiosyncratic forces unique to each situation. A deeper understanding of these factors and how they interact is critical for the development of strategies to teach and evaluate professionalism in practice.

Faculty staff perceptions of feedback to residents after direct observation of clinical skills.

CONTEXT: Performance-based workplace assessments are increasingly important in clinical training. Given the inaccuracy of self-assessment, the provision of external feedback to residents about their clinical skills is necessary for the development of expertise. However, little is known about the processes used by faculty members in giving feedback to residents after observing them with patients. This study explores the factors that underpin faculty members' decisions regarding the feedback they give to residents after directly observing them with patients and the factors that influence how feedback is delivered. METHODS: In 2009, 44 general internal medicine faculty staff responsible for out-patient resident teaching from 16 internal medicine residency programmes watched four videotaped scenarios and two live scenarios of standardised residents (SRs) with standardised patients and rated the SRs using the mini-clinical evaluation exercise (mini-CEX) format. Faculty staff also provided feedback to the SRs after the live encounters. After each encounter, faculty staff were individually interviewed using a semi-structured interview. Interviews were videotaped, transcribed and analysed using grounded theory methods. RESULTS: Two broad themes were identified in faculty members' descriptions of the feedback process: variability in feedback techniques, and the factors that influence how faculty staff think and feel about delivering feedback. Multiple approaches to feedback delivery were observed. Faculty members' tensions in balancing positive and negative feedback, their own perceived self-efficacy, their perceptions of the resident's insight, receptivity, skill and potential, the faculty member-resident relationship and contextual factors impacted the feedback process. CONCLUSIONS: The provision of feedback by faculty staff to residents after observing resident-patient interactions is a complex and dynamic process and is influenced by many factors. Understanding these cognitive and affective factors may provide insight into potential new approaches to faculty development to improve faculty staff's feedback skills and the effectiveness of their feedback.

Opening the black box of clinical skills assessment via observation: a conceptual model.

OBJECTIVES: This study was intended to develop a conceptual framework of the factors impacting on faculty members' judgements and ratings of resident doctors (residents) after direct observation with patients. METHODS: In 2009, 44 general internal medicine faculty members responsible for out-patient resident teaching in 16 internal medicine residency programmes in a large urban area in the eastern USA watched four videotaped scenarios and two live scenarios of standardised residents engaged in clinical encounters with standardised patients. After each, faculty members rated the resident using a mini-clinical evaluation exercise and were individually interviewed using a semi-structured interview. Interviews were videotaped, transcribed and analysed using grounded theory methods. RESULTS: Four primary themes that provide insights into the variability of faculty assessments of residents' performance were identified: (i) the frames of reference used by faculty members when translating observations into judgements and ratings are variable; (ii) high levels of inference are used during the direct observation process; (iii) the methods by which judgements are synthesised into numerical ratings are variable, and (iv) factors external to resident performance influence ratings. From these themes, a conceptual model was developed to describe the process of observation, interpretation, synthesis and rating. CONCLUSIONS: It is likely that multiple factors account for the variability in faculty ratings of residents. Understanding these factors informs potential new approaches to faculty development to improve the accuracy, reliability and utility of clinical skills assessment.

The roles of practice systems and individual effort in quality performance.

INTRODUCTION: Individual effort and practice systems contribute to quality performance, but the nature of their contributions remains unclear. METHODS: This study assessed the roles of individual attributes and behaviours versus practice attributes in quality performance by assessing general internists' perceptions of factors that drive their engagement in quality improvement (QI). The authors interviewed 20 physicians in two distinct categories from diverse practice settings who had the greatest discordance between their ranked scores on standardised measures of individual quality performance and practice 'systems' performance. RESULTS: Findings suggest that there are subtle but important differences between high-scoring physicians practising in low-scoring practice systems, and low-scoring physicians practising in high-scoring practice systems with regards to quality performance and improvement. Physicians with high individual and low systems scores contributed a greater individual effort in quality improvement (QI), exhibited greater internal drivers to change, and reported a greater number and broader list of QI activities than physicians with low individual scores and high systems scores. Physicians with high individual scores also tended to be more reflective. There was a lack of consensus between categories on the relative usefulness of different systems resources, including electronic information systems. Our findings also suggest that physicians practice in isolation and autonomously, and highly independent of each other, and perceive a tension between pursuing technical quality and patient satisfaction at the same time. Both categories were skeptical of performance measurement more generally. CONCLUSION: QI efforts may be more effective if they foster both specific individual attitudes and capabilities, as well as improve practice-level systems.