Longitudinal Assessment of Resident Performance Using Entrustable Professional Activities.

Importance: Entrustable professional activities (EPAs) are an emerging workplace-based, patient-oriented assessment approach with limited empirical evidence. Objective: To measure the development of pediatric trainees' clinical skills over time using EPA-based assessment data. Design, Setting, and Participants: Prospective cohort study of categorical pediatric residents over 3 academic years (2015-2016, 2016-2017, and 2017-2018) assessed on 17 American Board of Pediatrics EPAs. Residents in training at 23 pediatric residency programs in the Association of Pediatric Program Directors Longitudinal Educational Assessment Research Network were included. Assessment was conducted by clinical competency committee members, who made summative assessment decisions regarding levels of supervision required for each resident and each EPA. Data were collected from May 2016 to November 2018 and analyzed from November to December 2018. Interventions: Longitudinal, prospective assessment using EPAs. Main Outcomes and Measures: Trajectories of supervision levels by EPA during residency training and how often graduating residents were deemed ready for unsupervised practice in each EPA. Results: Across the 5 data collection cycles, 1987 residents from all 3 postgraduate years in 23 residency programs were assigned 25 503 supervision level reports for the 17 general pediatrics EPAs. The 4 EPAs that required the most supervision across training were EPA 14 (quality improvement) on the 5-level scale (estimated mean level at graduation, 3.7; 95% CI, 3.6-3.7) and EPAs 8 (transition to adult care; mean, 7.0; 95% CI, 7.0-7.1), 9 (behavioral and mental health; mean, 6.6; 95% CI, 6.5-6.6), and 10 (resuscitate and stabilize; mean, 6.9; 95% CI, 6.8-7.0) on the expanded 5-level scale. At the time of graduation (36 months), the percentage of trainees who were rated at a supervision level corresponding to "unsupervised practice" varied by EPA from 53% to 98%. If performance standards were set to align with 90% of trainees achieving the level of unsupervised practice, this standard would be met for only 8 of the 17 EPAs (although 89% met this standard for EPA 17, performing the common procedures of the general pediatrician). Conclusions and Relevance: This study presents initial evidence for empirically derived practice readiness and sets the stage for identifying curricular gaps that contribute to discrepancy between observed practice readiness and standards needed to produce physicians able to meet the health needs of the patient populations they serve. Future work should compare these findings with postgraduation outcomes data as a means of seeking validity evidence.

Senior student goal setting during subinternship: a mixed-methods study.

BACKGROUND: New developments in Subinternship curricula attempt to establish self-directed learning skills that will translate into the 1st year of residency. However, we know little about how well senior medical students' self-directed learning goals match with what is expected of them in residency. PURPOSES: We designed a mixed-methods study to examine the goals set by senior students at Georgia Health Sciences University on Pediatric Subinternship and to determine how those goals relate to the 6 Accreditation Council for Graduate Medical Education (ACGME) competencies. METHODS: We used an iterative process to categorize self-directed learning goals on Pediatric Subinternships (n=188 goals generated by 57 senior students) by (a) the 6 ACGME competencies and (b) general or specific goals. We used tests of association and multivariate modeling to compare goal categories by clinical site and time of year. RESULTS: There were 56.3% of goals addressing patient care. Professionalism and systems-based practice goals were rare. Of the goals, 72% were general, but goals written by students at the newborn nursery and neonatal intensive care unit sites were significantly more likely to be specific than goals written by students on the general inpatient subinternship. CONCLUSIONS: The overwhelming majority of senior medical student goals on a Pediatric Subinternship are general and related to patient care. Students may need assistance with developing more specific goals in all 6 competencies. Our findings suggest that understanding and incorporating students' learning goals may be important for enhancing the potential effectiveness of transition-to-residence curricula.

Defining medical professionalism: a qualitative study.

BACKGROUND: Elements of professionalism are well-described in the literature and medical schools continue to struggle with how to teach these concepts effectively. PURPOSE: The purpose of this study was to investigate the meaning of medical professionalism to medical students, residents, academic faculty and patients and to determine areas of congruence and difference. METHODS: In this qualitative study we conducted 8 focus groups to discover subjects' beliefs, perceptions and expectations of medical professionals. Sessions were audiotaped and transcribed, and themes identified through an immersion/crystallisation process. Concept maps were prepared to aid understanding. RESULTS: Recurring primary themes of knowledge/technical skills, patient relationship and character virtues were identified. Secondary themes were medicine as a unique profession, personal congruence and the importance of peer relationships. There was a shift in emphasis reflecting differing stages in the learner continuum. Although patients desired skilled technicians, their themes focused on relationships. Several unique themes were also identified. CONCLUSIONS: Some elements of professionalism are embraced by learners at all stages and by patients. Notably, when compared to components of the American Board of Internal Medicine Physician Charter, themes relating to social justice elements were lacking. Differences in emphasis by learner groups reflect the inherent challenges to teaching professionalism successfully. Future studies investigating these differing perceptions are needed to help clarify our teaching mission.