Management of the Electronic Health Record Inbox: Results From a National Survey of Internal Medicine Program Directors.

Background Internal medicine (IM) resident physicians spend a considerable amount of time managing their inbox as part of their longitudinal continuity clinic experience. There are no standardized guidelines for how programs should train, monitor, or supervise residents in this type of patient care. Objective To understand how IM residency programs educate, monitor, and supervise resident electronic health record (EHR) inbox management as part of their longitudinal continuity clinic and determine whether patient safety events have occurred due to EHR inbox-related patient care decisions made by unsupervised resident physicians. Methods In August 2021, 439 program directors at accredited US IM residency programs who were members of the Association of Program Directors in Internal Medicine (APDIM) were asked 12 questions developed by the study authors and APDIM survey committee members regarding resident EHR inbox management as part of the annual APDIM survey. Results Two hundred and sixty-seven (61%) PDs responded. The majority (224 of 267, 84%) of programs provided guidelines for expected message response times; less than half (115, 43%) monitored timeliness metrics. Only half (135; 51%) of programs required faculty supervision of inbox messages for all residents; 28% (76) did not require supervision for any residents. Twenty-one percent of PDs (56) reported awareness of a patient safety event occurring due to an unsupervised resident inbox-related patient care decision. Conclusions Substantial variability exists in how IM residency programs train, monitor, supervise, and provide coverage for resident inbox work. Program directors are aware of patient safety events resulting from unsupervised resident inbox management.

Does Increased Schedule Flexibility Lead to Change? A National Survey of Program Directors on 2017 Work Hours Requirements.

BACKGROUND: The learning and working environment for resident physicians shifted dramatically over the past two decades, with increased focus on work hours, resident wellness, and patient safety. Following two multi-center randomized trials comparing 16-h work limits for PGY-1 trainees to more flexible rules, the ACGME implemented new flexible work hours standards in 2017. OBJECTIVE: We sought to determine program directors' (PDs) support for the work hour changes and programmatic response. DESIGN: In 2017, US Internal Medicine PDs were surveyed about their degree of support for extension of PGY-1 work hour limits, whether they adopted the new maximum continuous work hours permitted, and reasons for their decisions. KEY RESULTS: The response rate was 70% (266/379). Fifty-seven percent of PDs (n = 151) somewhat/strongly support the new work hour rules for PGY-1 residents, while only 25% of programs (N = 66) introduced work periods greater than 16-h on any rotation. Higher rates of adopting change were seen in PDs who strongly/somewhat supported the change (56/151 [37%], P < 0.001), had tenure of 6+ years (33/93 [35%], P = 0.005), were of non-general internal medicine subspecialty (30/80 [38%], P = 0.003), at university-based programs (35/101 [35%], P = 0.009), and with increasing number of approved positions (< 38, 10/63 [16%]; 38-58, 13/69 [19%]; 59-100, 15/64 [23%]; > 100, 28/68 [41%], P = 0.005). Areas with the greatest influence for PDs not extending work hours were the 16-h rule working well (56%) and risk to PGY1 well-being (47%). CONCLUSIONS: Although the majority of PDs support the ACGME 2017 work hours rules, only 25% of programs made immediate changes to extend hours. These data reveal that complex, often competing, forces influence PDs' decisions to change trainee schedules.

Assessment of Training in Health Disparities in US Internal Medicine Residency Programs.

Importance: Health disparities continue to exist despite the call to increase education of health care practitioners. An assessment of health disparities education has not been previously studied in a national cohort. Objective: To describe and compare the curriculum on health disparities from the perspective of program directors and perceptions of training among internal medicine residents. Design, Setting, and Participants: This cross-sectional survey study used a survey of US internal medicine program directors, the 2015 Association of Program Directors in Internal Medicine annual survey, which included questions about health disparities curriculum, and a 1-time survey of US internal medicine residents that asked questions related to their training in health disparities on the American College of Physicians 2015 Internal Medicine In-Training Examination. All internal medicine program directors who were members of Association of Program Directors in Internal Medicine (368 of 396 accredited programs), and internal medicine residents who took the Internal Medicine In-Training Examination were eligible. Final analysis of the merged data set was completed in 2018. Exposures: Questions were included on the annual Association of Program Directors in Internal Medicine survey and the Internal Medicine In-Training Examination. Main Outcomes and Measures: Program directors reported on presence of health disparities curriculum, educational methods, quality, barriers and challenges to curriculum, and Clinical Learning Environment Review. Residents reported whether they received training and quality of the training in health disparities. Results: A total of 227 program directors (response rate, 61.7%) and 22 723 residents (response rate, 87.2%) responded to the surveys. A total of 90 program directors (39.6%) reported a curriculum in health disparities, but among these, only 16 program directors (17.8%) felt quality of their education was very good or excellent. In more than half of the programs (52 programs [55.9%]), outcomes of the curriculum were not measured. After merging, the combined data set included 18 883 residents from 366 APDIM member programs with 225 program director responses. Among these, 13 251 residents (70.2%) reported some training in caring for patients at risk for health disparities. Of residents who reported receiving training, 10 494 (79.2%) rated the quality as very good or excellent. Conclusions and Relevance: These findings suggest that that despite the Clinical Learning Environment Review mandate and Accreditation Council for Graduate Medical Education required competencies for training in health disparities, relatively few internal medicine programs in the US provided educational curriculum. Additionally, the existence of health disparities curricula in internal medicine training programs was not associated with resident's perception of training or its quality.

Associations Among Practice Variation, Clinician Characteristics, and Care Algorithm Usage: A Multispecialty Vignette Study.

The objective was to quantitatively evaluate clinician characteristics associated with unwarranted practice variation, and how clinical care algorithms influence this variation. Participants (142 physicians, 53 nurse practitioners, and 9 physician assistants in family medicine, internal medicine, and cardiology) described their management of 4 clinical vignettes, first based on their own practice (unguided), then using care algorithms (guided). The authors quantitatively estimated variation in management. Cardiologists demonstrated 17% lower variation in unguided responses than generalists (fold-change 0.83 [95% confidence interval (CI) 0.68, 0.97]), and those who agreed that practice variation can realistically be reduced had 16% lower variation than those who did not (fold-change 0.84 [CI, 0.71, 0.99]). A 17% reduction in variation was observed for guided responses compared with baseline (unguided) responses (fold-change 0.83 [CI, 0.76, 0.90]). Differences were otherwise similar across clinician subgroups and attitudes. Unwarranted practice variation was similar across most clinician subgroups. The authors conclude that care algorithms can reduce variation in management.

Awareness of Bullying in Residency: Results of a National Survey of Internal Medicine Program Directors.

BACKGROUND: Bullying of medical trainees is believed to occur more frequently in medical education than once thought. OBJECTIVE: We conducted a survey to understand internal medicine program director (PD) perspectives and awareness about bullying in their residency programs. METHODS: The 2015 Association of Program Directors in Internal Medicine (APDIM) annual survey was e-mailed to 368 of 396 PDs with APDIM membership, representing 93% of internal medicine residency programs. Questions about bullying were embedded within the survey. Bivariate analyses were performed on PD and program characteristics. RESULTS: Of a total of 368 PD APDIM members, 227 PDs (62%) responded to the survey. Less than one-third of respondents (71 of 227, 31%) reported being aware of bullying in their residency programs during the previous year. There were no significant differences in program or PD characteristics between respondents who reported bullying in their programs and those who did not (gender, tenure as PD, geographic location, or specialty, all P > .05). Those who acknowledged bullying in their program were more likely to agree it was a problem in graduate medical education (P < .0001), and it had a significant negative impact on the learning environment (P < .0001). The majority of reported events entailed verbal disparagements, directed toward interns and women, and involved attending physicians, other residents, and nurses. CONCLUSIONS: This national survey of internal medicine PDs reveals that a minority of PDs acknowledged recent bullying in their training programs, and reportedly saw it as a problem in the learning environment.

Practice variation and practice guidelines: Attitudes of generalist and specialist physicians, nurse practitioners, and physician assistants.

OBJECTIVE: To understand clinicians' beliefs about practice variation and how variation might be reduced. METHODS: We surveyed board-certified physicians (N = 178), nurse practitioners (N = 60), and physician assistants (N = 12) at an academic medical center and two community clinics, representing family medicine, general internal medicine, and cardiology, from February-April 2016. The Internet-based questionnaire ascertained clinicians' beliefs regarding practice variation, clinical practice guidelines, and costs. RESULTS: Respondents agreed that practice variation should be reduced (mean [SD] 4.5 [1.1]; 1 = strongly disagree, 6 = strongly agree), but agreed less strongly (4.1 [1.0]) that it can realistically be reduced. They moderately agreed that variation is justified by situational differences (3.9 [1.2]). They strongly agreed (5.2 [0.8]) that clinicians should help reduce healthcare costs, but agreed less strongly (4.4 [1.1]) that reducing practice variation would reduce costs. Nearly all respondents (234/249 [94%]) currently depend on practice guidelines. Clinicians rated differences in clinician style and experience as most influencing practice variation, and inaccessibility of guidelines as least influential. Time to apply standards, and patient decision aids, were rated most likely to help standardize practice. Nurse practitioners and physicians assistants (vs physicians) and less experienced (vs senior) clinicians rated more favorably several factors that might help to standardize practice. Differences by specialty and academic vs community practice were small. CONCLUSIONS: Clinicians believe that practice variation should be reduced, but are less certain that this can be achieved. Accessibility of guidelines is not a significant barrier to practice standardization, whereas more time to apply standards is viewed as potentially helpful.

Flipped Classrooms in Graduate Medical Education: A National Survey of Residency Program Directors.

PURPOSE: To begin to quantify and understand the use of the flipped classroom (FC)-a progressive, effective, curricular model-in internal medicine (IM) education in relation to residency program and program director (PD) characteristics. METHOD: The authors conducted a survey that included the Flipped Classroom Perception Instrument (FCPI) in 2015 regarding programs' use and PDs' perceptions of the FC model. RESULTS: Among the 368 IM residency programs, PDs at 227 (61.7%) responded to the survey and 206 (56.0%) completed the FCPI. Regarding how often programs used the FC model, 34 of the 206 PDs (16.5%) reported "never"; 44 (21.4%) reported "very rarely"; another 44 (21.4%) reported "somewhat rarely"; 59 (28.6%) reported "sometimes"; 16 (7.8%) reported "somewhat often"; and 9 (4.4%) reported "very often." The mean FCPI score (standard deviation [SD]) for the in-class application factor (4.11 [0.68]) was higher (i.e., more favorable) than for the preclass activity factor (3.94 [0.65]) (P < .001). FC perceptions (mean [SD]) were higher among younger PDs (≤ 50 years, 4.12 [0.62]; > 50 years, 3.94 [0.61]; P = .04) and women compared with men (4.28 [0.56] vs. 3.91 [0.62]; P < .001). PDs with better perceptions of FCs had higher odds of using FCs (odds ratio, 4.768; P < .001). CONCLUSIONS: Most IM programs use the FC model at least to some extent, and PDs prefer the interactive in-class components over the independent preclass activities. PDs who are women and younger perceived the model more favorably.

E-learning in graduate medical education: survey of residency program directors.

BACKGROUND: E-learning-the use of Internet technologies to enhance knowledge and performance-has become a widely accepted instructional approach. Little is known about the current use of e-learning in postgraduate medical education. To determine utilization of e-learning by United States internal medicine residency programs, program director (PD) perceptions of e-learning, and associations between e-learning use and residency program characteristics. METHODS: We conducted a national survey in collaboration with the Association of Program Directors in Internal Medicine of all United States internal medicine residency programs. RESULTS: Of the 368 PDs, 214 (58.2%) completed the e-learning survey. Use of synchronous e-learning at least sometimes, somewhat often, or very often was reported by 85 (39.7%); 153 programs (71.5%) use asynchronous e-learning at least sometimes, somewhat often, or very often. Most programs (168; 79%) do not have a budget to integrate e-learning. Mean (SD) scores for the PD perceptions of e-learning ranged from 3.01 (0.94) to 3.86 (0.72) on a 5-point scale. The odds of synchronous e-learning use were higher in programs with a budget for its implementation (odds ratio, 3.0 [95% CI, 1.04-8.7]; P = .04). CONCLUSIONS: Residency programs could be better resourced to integrate e-learning technologies. Asynchronous e-learning was used more than synchronous, which may be to accommodate busy resident schedules and duty-hour restrictions. PD perceptions of e-learning are relatively moderate and future research should determine whether PD reluctance to adopt e-learning is based on unawareness of the evidence, perceptions that e-learning is expensive, or judgments about value versus effectiveness.

Important Skills for Internship and the Fourth-Year Medical School Courses to Acquire Them: A National Survey of Internal Medicine Residents.

PURPOSE: To obtain feedback from internal medicine residents, a key stakeholder group, regarding both the skills needed for internship and the fourth-year medical school courses that prepared them for residency. This feedback could inform fourth-year curriculum redesign efforts. METHOD: All internal medicine residents taking the 2013-2014 Internal Medicine In-Training Examination were asked to rank the importance of learning 10 predefined skills prior to internship and to use a dropdown menu of 11 common fourth-year courses to rank the 3 most helpful in preparing for internship. The predefined skills were chosen based on a review of the literature, a national subinternship curriculum, and expert consensus. Chi-square statistics were used to test for differences in responses between training levels. RESULTS: Of the 24,820 internal medicine residents who completed the exam, 20,484 (83%) completed the survey, had complete identification numbers, and consented to have their responses used for research. The three skills most frequently rated as very important were identifying when to seek additional help and expertise, prioritizing clinical tasks and managing time efficiently, and communicating with other providers around care transitions. The subinternship/acting internship was most often selected as being the most helpful course in preparing for internship. CONCLUSIONS: These findings indicate which skills and fourth-year medical school courses internal medicine residents found most helpful in preparing for internship and confirm the findings of prior studies highlighting the perceived value of subinternships. Internal medicine residents and medical educators agree on the skills students should learn prior to internship.

Rating the Quality of Entrustable Professional Activities: Content Validation and Associations with the Clinical Context.

BACKGROUND: Entrustable professional activities (EPAs) have been developed to assess resident physicians with respect to Accreditation Council for Graduate Medical Education (ACGME) competencies and milestones. Although the feasibility of using EPAs has been reported, we are unaware of previous validation studies on EPAs and potential associations between EPA quality scores and characteristics of educational programs. OBJECTIVES: Our aim was to validate an instrument for assessing the quality of EPAs for assessment of internal medicine residents, and to examine associations between EPA quality scores and features of rotations. DESIGN: This was a prospective content validation study to design an instrument to measure the quality of EPAs that were written for assessing internal medicine residents. PARTICIPANTS: Residency leadership at Mayo Clinic, Rochester participated in this study. This included the Program Director, Associate program directors and individual rotation directors. INTERVENTIONS: The authors reviewed salient literature. Items were developed to reflect domains of EPAs useful for assessment. The instrument underwent further testing and refinement. Each participating rotation director created EPAs that they felt would be meaningful to assess learner performance in their area. These 229 EPAs were then assessed with the QUEPA instrument to rate the quality of each EPA. MAIN MEASURES: Performance characteristics of the QUEPA are reported. Quality ratings of EPAs were compared to the primary ACGME competency, inpatient versus outpatient setting and specialty type. KEY RESULTS: QUEPA tool scores demonstrated excellent reliability (ICC range 0.72 to 0.94). Higher ratings were given to inpatient versus outpatient (3.88, 3.66; p = 0.03) focused EPAs. Medical knowledge EPAs scored significantly lower than EPAs assessing other competencies (3.34, 4.00; p < 0.0001). CONCLUSIONS: The QUEPA tool is supported by good validity evidence and may help in rating the quality of EPAs developed by individual programs. Programs should take care when writing EPAs for the outpatient setting or to assess medical knowledge, as these tended to be rated lower.

Hospitalist career decisions among internal medicine residents.

BACKGROUND: Hospital medicine is a rapidly growing field of internal medicine. However, little is known about internal medicine residents' decisions to pursue careers in hospital medicine (HM). OBJECTIVE: To identify which internal medicine residents choose a career in HM, and describe changes in this career choice over the course of their residency education. DESIGN: Observational cohort using data collected from the annual Internal Medicine In-Training Examination (IM-ITE) survey. PARTICIPANTS: 16,781 postgraduate year 3 (PGY-3) North American internal medicine residents who completed the annual IM-ITE survey in 2009-2011, 9,501 of whom completed the survey in all 3 years of residency. MAIN MEASURES: Self-reported career plans for individual residents during their postgraduate year 1 (PGY-1), postgraduate year 2 (PGY-2) and PGY-3. KEY RESULTS: Of the 16,781 graduating PGY-3 residents, 1,552 (9.3 %) reported HM as their ultimate career choice. Of the 951 PGY-3 residents planning a HM career among the 9,501 residents responding in all 3 years, 128 (13.5 %) originally made this decision in PGY-1, 192 (20.2 %) in PGY-2, and 631 (66.4 %) in PGY-3. Only 87 (9.1 %) of these 951 residents maintained a career decision of HM during all three years of residency education. CONCLUSIONS: Hospital medicine is a reported career choice for an important proportion of graduating internal medicine residents. However, the majority of residents do not finalize this decision until their final year.

5 ways statistics can fool you--tips for practicing clinicians.

Published literature suggests that many clinicians are not fully equipped to evaluate and apply research reports for the care of their patients. In this article, we introduce and illustrate five basic statistical concepts that can significantly impact the interpretation of the medical literature and its application to the care of patients, drawing examples from the vaccine literature: (i) consider clinical and statistical significance separately, (ii) evaluate absolute risks rather than relative risks, (iii) examine confidence intervals rather than p values, (iv) use caution when considering isolated significant p values in the setting of multiple testing, and (v) keep in mind that statistically nonsignificant results may not exclude clinically important benefits or harms. These tips may help busy clinicians better interpret the increasingly overwhelming amount of medical literature they are faced with in their daily practices.

Update in internal medicine residency education: a review of the literature in 2010 and 2011.

BACKGROUND: Evidence-based practice in education requires high-quality evidence, and many in the medical education community have called for an improvement in the methodological quality of education research. OBJECTIVE: Our aim was to use a valid measure of medical education research quality to highlight the methodological quality of research publications and provide an overview of the recent internal medicine (IM) residency literature. METHODS: We searched MEDLINE and PreMEDLINE to identify English-language articles published in the United States and Canada between January 1, 2010, and December 31, 2011, focusing on IM residency education. Study quality was assessed using the Medical Education Research Study Quality Instrument (MERSQI), which has demonstrated reliability and validity. Qualitative articles were excluded. Articles were ranked by quality score, and the top 25% were examined for common themes, and 2 articles within each theme were selected for in-depth presentation. RESULTS: The search identified 731 abstracts of which 223 articles met our inclusion criteria. The mean (±SD) MERSQI score of the 223 studies included in the review was 11.07 (±2.48). Quality scores were highest for data analysis (2.70) and lowest for study design (1.41) and validity (1.29). The themes identified included resident well-being, duty hours and resident workload, career decisions and gender, simulation medicine, and patient-centered outcomes. CONCLUSIONS: Our review provides an overview of the IM medical education literature for 2010-2011, highlighting 5 themes of interest to the medical education community. Study design and validity are 2 areas where improvements in methodological quality are needed, and authors should consider these when designing research protocols.

General medicine vs subspecialty career plans among internal medicine residents.

CONTEXT: Current medical training models in the United States are unlikely to produce sufficient numbers of general internists and primary care physicians. Differences in general internal medicine (GIM) career plans between internal medicine residency program types and across resident demographics are not well understood. OBJECTIVE: To evaluate the general medicine career plans of internal medicine residents and how career plans evolve during training. DESIGN, SETTING, AND PARTICIPANTS: A study of US internal medicine residents using an annual survey linked to the Internal Medicine In-Training Examination taken in October of 2009-2011 to evaluate career plans by training program, sex, and medical school location. Of 67,207 US eligible categorical and primary care internal medicine residents, 57,087 (84.9%) completed and returned the survey. Demographic data provided by the National Board of Medical Examiners were available for 52,035 (77.4%) of these residents, of whom 51,390 (76.5%) responded to all survey items and an additional 645 (1.0%) responded to at least 1 survey item. Data were analyzed from the 16,781 third-year residents (32.2%) in this sample. MAIN OUTCOME MEASURES: Self-reported ultimate career plans of internal medicine residents. RESULTS: A GIM career plan was reported by 3605 graduating residents (21.5%). A total of 562 primary care program (39.6%) and 3043 categorical (19.9%) residents reported GIM as their ultimate career plan (adjusted odds ratio [AOR], 2.76; 99% CI, 2.35-3.23; P < .001). Conversely, 10 008 categorical (65.3%) and 745 primary care program (52.5%) residents reported a subspecialty career plan (AOR, 1.90; 99% CI, 1.62-2.23; P < .001). GIM career plans were reported more frequently by women than men (26.7% vs 17.3%, respectively; AOR, 1.69; 99% CI, 1.53-1.87; P < .001). US medical graduates were slightly more likely to report GIM career plans than international medical graduates (22.0% vs 21.1%, respectively; AOR, 1.76; 99% CI, 1.50-2.06; P < .001). Within primary care programs, US medical graduates were much more likely to report GIM career plans than international medical graduates (57.3% vs 27.3%, respectively; AOR, 3.48; 99% CI, 2.58-4.70; P < .001). Compared with their counterparts, maintaining a first-year GIM career plan over the course of their training was more likely among primary care program residents (68.2% vs 52.3%; AOR, 1.81; 99% CI, 1.25-2.64; P < .001), women (62.4% vs 47.2%; AOR, 1.75; 99% CI, 1.34-2.29; P < .001), and US medical graduates (60.9% vs 49.2%; AOR, 1.48; 99% CI, 1.13-1.93; P < .001). CONCLUSION: Reported GIM career plans were markedly less common than subspecialty career plans among internal medicine residents, including those in primary care training programs, and differed according to resident sex, medical school location, and program type.

Retail clinic visits: are resident paneled patients more likely to make multiple visits?

BACKGROUND AND OBJECTIVES: Primary care resident physicians generally have significantly fewer hours in outpatient clinic per week than staff physicians. With the transient nature of a primary care resident's practice, do the patients of residents utilize the health care system differently? Our hypothesis was that patients paneled with a resident physician would be more likely to utilize retail clinics with repeat week day visits than those patients paneled with a staff primary care physician. METHODS: A retrospective review of 16,318 retail clinic visits made in 2009 was studied. Since the patient making a repeat retail clinic visit had experience with the services available, they were making an informed decision on the type of care desired (their primary care provider or not). The patients who made repeat retail clinical visits were divided into groups by age, those younger than 18 years and those 18 years and older. RESULTS: Via multiple logistic regression, patients under 18 years demonstrated that a patient paneled with a resident physician was no more likely to have repeat visits to the retail clinic than a staff physician's patient, when controlling for all other variables. Similarly, those ages 18 years and older also demonstrated no differences in resident or staff paneled patients in utilization of retail clinics during the work day. CONCLUSIONS: Multivariate analysis found no difference in the utilization of repeat retail clinic visits during 2009 with resident paneled patients as compared to staff physician patients.