The relationship between grit and selected personality measures in medical students.

OBJECTIVES: To test the hypothesis that scores on a Grit scale are positively associated with personality measures that are conducive to relationship building (Empathy, Self-Esteem, Activity, and Sociability), but inversely associated with personality measures that are detrimental to interpersonal relationships (Neuroticism-Anxiety, Aggression-Hostility, Impulsive Sensation Seeking, and Loneliness). METHODS: Convenient sampling was used that included 241 medical students at Sidney Kimmel Medical College at Thomas Jefferson University who participated in this ex post facto research. Validated instruments were used to measure Grit, Empathy, Self-Esteem, Activity, Sociability, Neuroticism-Anxiety, Aggression-Hostility, Impulsive Sensation Seeking, and Loneliness. Bivariate correlations and multivariate regression were used to examine relationships between scores on the Grit scale and personality measures. RESULTS: Results of bivariate correlational analyses showed that scores on the Grit scale were positively and significantly (p<0.01) correlated with measures of Self-Esteem (r=0.35), Empathy (r=0.26), and Activity (r=0.17); but negatively and significantly (p<0.01) correlated with measures of Loneliness (r=-0.28), Aggression-Hostility (r=-0.23), Neuroticism-Anxiety (r=-0.22), and Impulsive Sensation Seeking (r=-0.18). Regression analysis indicated that in a multivariate model, higher scores on Self-Esteem and Empathy and lower scores on Aggression-Hostility were uniquely and significantly associated with Grit scores (R=0.43, p<0.01). CONCLUSIONS: Research hypothesis was partially confirmed, suggesting that medical students with higher Grit scores were likely to have higher empathic orientation in patient care and greater Self-Esteem. Conversely, those with higher degrees of Grit displayed lower levels of Aggression-Hostility and Impulsive Sensation Seeking. The Implications of these findings for medical education are discussed.

Identifying Students at Risk of Failing the USMLE Step 2 Clinical Skills Examination.

BACKGROUND AND OBJECTIVES: New standards announced in 2017 could increase the failure rate for Step 2 Clinical Skills (CS). The purpose of this study was to identify student performance metrics associated with risk of failing. METHODS: Data for 1,041 graduates of one medical school from 2014 through 2017 were analyzed, including 30 (2.9%) failures. Metrics included Medical College Admission Test, United States Medical Licensing Examination Step 1, and clerkship National Board of Medical Examiners (NBME) Subject Examination scores; faculty ratings in six clerkships; and scores on an objective structured clinical examination (OSCE). Bivariate statistics and regression were used to estimate risk of failing. RESULTS: Those failing had lower Step 1 scores, NBME scores, faculty ratings, and OSCE scores (P<.02). Students with four or more low ratings were more likely to fail compared to those with fewer low ratings (relative risk [RR], 12.76, P<.0001). Logistic regression revealed other risks: low surgery NBME scores (RR 3.75, P=.02), low pediatrics NBME scores (RR 3.67, P=.02), low ratings in internal medicine (RR 3.42, P=.004), and low OSCE Communication/Interpersonal Skills (RR 2.55, P=.02). CONCLUSIONS: Certain medical student performance metrics are associated with risk of failing Step 2 CS. It is important to clarify these and advise students accordingly.

Standardized patient assessment of medical student empathy: ethnicity and gender effects in a multi-institutional study.

PURPOSE: To examine, primarily, the effects of ethnicity and gender, which could introduce bias into scoring, on standardized patient (SP) assessments of medical students and, secondarily, to examine medical students' self-reported empathy for ethnicity and gender effects so as to compare self-perception with the perceptions of SPs. METHOD: Participants were 577 students from four medical schools in 2012: 373 (65%) were white, 79 (14%) black/African American, and 125 (22%) Asian/Pacific Islander. These students were assessed by 84 SPs: 62 (74%) were white and 22 (26%) were black/African American. SPs completed the Jefferson Scale of Patient Perceptions of Physician Empathy (JSPPPE) and the Global Ratings of Empathy tool. Students completed the Jefferson Scale of Empathy and two Interpersonal Reactivity Index subscales. The investigators used 2,882 student-SP encounters in their analyses. RESULTS: Analyses of SPs' assessments of students' empathy indicated significant interaction effects of gender and ethnicity. Female students, regardless of ethnicity, obtained significantly higher mean JSPPPE scores than men. Female black/African American, female white, and female Asian/Pacific Islander students scored significantly higher on the JSPPPE than their respective male counterparts. Male black/African American students obtained the lowest SP assessment scores of empathy regardless of SP ethnicity. Black/African American students obtained the highest mean scores on self-reported empathy. CONCLUSIONS: The significant interaction effects of ethnicity and gender in clinical encounters, plus the inconsistencies observed between SPs' assessments of students' empathy and students' self-reported empathy, raise questions about possible ethnicity and gender biases in the SPs' assessments of medical students' clinical skills.

Evaluation of the validity of medical students' self-assessments of proficiency in clinical simulations.

BACKGROUND: The accuracy of self-assessments has not been well supported in the literature. This study was undertaken to examine the validity of medical students' ratings of their proficiency during encounters with simulated patients and simulation devices. METHODS: Confidential self-assessments for 10 skills were collected from 195 students during a formal clinical skills assessment related to 3 cases at the end of a surgery clerkship. The cases required students to gather data from simulated patients and perform procedures such as rectal examinations, nasogastric tube insertions, and suturing on bench simulation models. The patients were trained to assess student performance. RESULTS: There were significant differences between student self-assessments and simulated patient scores for general clinical skills as opposed to procedural skills. Students' mean self-assessments in the data gathering and interpersonal skills were 2-6 % points higher than ratings of their proficiency by simulated patients. However, self-assessments on procedures were 5-8 points lower than patient ratings. The median correlation between self-assessments and patient ratings for general clinical skills such as data gathering and interpersonal skills was 0.08 (not significant), whereas the median correlation between student and patient ratings in procedures was 0.22 (P < 0.01). CONCLUSIONS: Third-year medical students' self-assessments for specific procedures are more valid than self-assessments of general clinical skills. Students are less confident in their procedural skills compared with general clinical skills. Although self-assessments should not be used as the sole measure of performance in clinical simulations, self-assessments for specific procedures can provide supplemental information on proficiency.

Medical students' self-reported empathy and simulated patients' assessments of student empathy: an analysis by gender and ethnicity.

PURPOSE: To examine the contribution of students' gender and ethnicity to assessments by simulated patients (SPs) of medical students' empathy, and to compare the results with students' self-assessments of their own empathy. METHOD: In 2008, the authors used three different tools to assess the empathy of 248 third-year medical students. Students completed the Jefferson Scale of Physician Empathy (JSPE), and SPs completed the Jefferson Scale of Patient Perceptions of Physician Empathy (JSPPPE) and a global rating of empathy (GRE) in 10 objective structured clinical examination (OSCE) encounters. RESULTS: Of the 248 students who completed an end-of-third-year OSCE, 176 (71%) also completed the JSPE. Results showed that women scored higher than men on all three measures of empathy. The authors detected no significant difference between white and Asian American students on their self-report JSPE scores. However, the SPs' assessments on the JSPPPE and on the GRE were significantly lower, indicating less empathy, for Asian American students. CONCLUSIONS: A tool for SPs to assess students' empathy during an OSCE could be helpful for unmasking some deficits in empathy in students during the third year of medical school. Because the authors found no significant differences on self-reported empathy, the differences they observed in the SPs' assessments of white and Asian American students were unexpected and need further exploration. These findings call for investigation into the reasons for such differences so that OSCEs and other examinations comply with the guidelines for fairness in educational and psychological testing as recommended by professional testing organizations.

A comparison of medical students' self-reported empathy with simulated patients' assessments of the students' empathy.

BACKGROUND: Empathy is necessary for communication between patients and physicians to achieve optimal clinical outcomes. AIM: To examine associations between Simulated Patients' (SPs) assessment of medical students' empathy and the students' self-reported empathy. METHODS: A total of 248 third-year medical students completed the Jefferson Scale of Physician Empathy (JSPE). SPs completed the Jefferson Scale of Patient Perceptions of Physician Empathy (JSPPPE), and a global rating of empathy in 10 objective clinical skills examination encounters during a comprehensive end of third-year clinical skills examination. RESULTS: High correlation was found between the scores on the JSPPPE and the global ratings of empathy completed by the SPs (r = 0.87, p < 0.01). A moderate but statistically significant correlation was observed between scores of the JSPE and the JSPPPE (r = 0.19, p < 0.05). Significant differences were observed on the JSPE and global ratings of empathy among top, middle and low scorers on the JSPPPE in the expected direction. CONCLUSIONS: While significant associations exist between students' self-reported scores on the JSPE and SPs' evaluations of students' empathy, the associations are not large enough to conclude that the two evaluations are redundant.

Measuring medical students' orientation toward lifelong learning: a psychometric evaluation.

BACKGROUND: The principle of lifelong learning is pervasive in regulations governing medical education and medical practice; yet, tools to measure lifelong learning are lagging in development. This study evaluates the Jefferson Scale of Physician Lifelong Learning (JeffSPLL) adapted for administration to medical students. METHOD: The Jefferson Scale of Physician Lifelong Learning-Medical Students (JeffSPLL-MS) was administered to 732 medical students in four classes. Factor analysis and t tests were performed to investigate its construct validity. RESULTS: Maximum likelihood factor analysis identified a three-factor solution explaining 46% of total variance. Mean scores of clinical and preclinical students were compared; clinical students scored significantly higher in orientation toward lifelong learning (P < .001). CONCLUSIONS: The JeffSPLL-MS presents findings consistent with key concepts of lifelong learning. Results from use of the JeffSPLL-MS may reliably inform curriculum design and education policy decisions that shape the careers of physicians.

The predictive validity of three versions of the MCAT in relation to performance in medical school, residency, and licensing examinations: a longitudinal study of 36 classes of Jefferson Medical College.

PURPOSE: The Medical College Admission Test (MCAT) has undergone several revisions for content and validity since its inception. With another comprehensive review pending, this study examines changes in the predictive validity of the MCAT's three recent versions. METHOD: Study participants were 7,859 matriculants in 36 classes entering Jefferson Medical College between 1970 and 2005; 1,728 took the pre-1978 version of the MCAT; 3,032 took the 1978-1991 version, and 3,099 took the post-1991 version. MCAT subtest scores were the predictors, and performance in medical school, attrition, scores on the medical licensing examinations, and ratings of clinical competence in the first year of residency were the criterion measures. RESULTS: No significant improvement in validity coefficients was observed for performance in medical school or residency. Validity coefficients for all three versions of the MCAT in predicting Part I/Step 1 remained stable (in the mid-0.40s, P < .01). A systematic decline was observed in the validity coefficients of the MCAT versions in predicting Part II/Step 2. It started at 0.47 for the pre-1978 version, decreased to between 0.42 and 0.40 for the 1978-1991 versions, and to 0.37 for the post-1991 version. Validity coefficients for the MCAT versions in predicting Part III/Step 3 remained near 0.30. These were generally larger for women than men. CONCLUSIONS: Although the findings support the short- and long-term predictive validity of the MCAT, opportunities to strengthen it remain. Subsequent revisions should increase the test's ability to predict performance on United States Medical Licensing Examination Step 2 and must minimize the differential validity for gender.

The devil is in the third year: a longitudinal study of erosion of empathy in medical school.

PURPOSE: This longitudinal study was designed to examine changes in medical students' empathy during medical school and to determine when the most significant changes occur. METHOD: Four hundred fifty-six students who entered Jefferson Medical College in 2002 (n = 227) and 2004 (n = 229) completed the Jefferson Scale of Physician Empathy at five different times: at entry into medical school on orientation day and subsequently at the end of each academic year. Statistical analyses were performed for the entire cohort, as well as for the "matched" cohort (participants who identified themselves at all five test administrations) and the "unmatched" cohort (participants who did not identify themselves in all five test administrations). RESULTS: Statistical analyses showed that empathy scores did not change significantly during the first two years of medical school. However, a significant decline in empathy scores was observed at the end of the third year which persisted until graduation. Findings were similar for the matched cohort (n = 121) and for the rest of the sample (unmatched cohort, n = 335). Patterns of decline in empathy scores were similar for men and women and across specialties. CONCLUSIONS: It is concluded that a significant decline in empathy occurs during the third year of medical school. It is ironic that the erosion of empathy occurs during a time when the curriculum is shifting toward patient-care activities; this is when empathy is most essential. Implications for retaining and enhancing empathy are discussed.

Short-, intermediate-, and long-term outcomes of Pennsylvania's continuum of tobacco education pilot project.

INTRODUCTION: The most effective time to introduce formal tobacco use treatment training for physicians is during the medical school experience. However, few medical schools have adopted standardized curricula, missing an important opportunity to influence future physician behavior. The Pennsylvania Continuum of Tobacco Education pilot project was undertaken from spring 2003 through summer 2005 to evaluate a generalizable method of improving students' knowledge, attitudes, and behaviors related to tobacco use treatment. METHODS: Intervention methods included a 1-day intensive multiformat seminar, followed by a reinforcement session 4 weeks later, within an internal medicine clerkship. Outcome measures included changes in students' attitudes, rates of "ask" and "advise" behaviors during clinical encounters, and performance on end-of-year clinical skills examinations. RESULTS: Short, intermediate, and long-term outcomes related to both smoking assessment and counseling improved as a result of the intervention. The percentage of students who obtained tobacco histories and counseled patients in clerkships increased following the seminar compared with the baseline. Nearly, all students demonstrated relevant skills during a clinical skills assessment at the end of the third year. DISCUSSION: The introduction of a standardized tobacco curriculum into medical school training is both feasible and effective. Results were sustained following the intervention, and the effects were reflected across several valid outcomes.

Components of postgraduate competence: analyses of thirty years of longitudinal data.

CONTEXT: The conceptualisation and measurement of competence in patient care are critical to the design of medical education programmes and outcome assessment. OBJECTIVE: We aimed to examine the major components and correlates of postgraduate competence in patient care. METHODS: A 24-item rating form with additional questions about resident doctors' performance and future residency offers was used. Study participants comprised 4560 subjects who graduated from Jefferson Medical College between 1975 and 2004. They pursued their graduate medical education in 508 hospitals. We used a longitudinal study design in which the rating form was completed by programme directors to evaluate residents at the end of the first postgraduate year. Factor analysis was used to identify the underlying components of postgraduate ratings. Multiple regression, t-test and correlational analyses were used to study the validity of the components that emerged. RESULTS: Two major components emerged, which we labelled 'Knowledge and Clinical Capabilities' and 'Professionalism', and which addressed the science and art of medicine, respectively. Performance measures during medical school, scores on medical licensing examinations, and global assessment of Medical Knowledge, Clinical Judgement and Data-gathering Skills showed higher correlations with scores on the Knowledge and Clinical Capabilities component. Global assessments of Professional Attitudes and ratings of Empathic Behaviour showed higher correlations with scores on the Professionalism component. Offers of continued residency and evaluations of desirable qualities were associated with both components. CONCLUSIONS: Psychometric support for measuring the components of Knowledge and Clinical Capabilities, and Professionalism provides an instrument to empirically evaluate educational outcomes to medical educators who are in search of such a tool.

Use of critical incidents to develop a rating form for resident evaluation of faculty teaching.

CONTEXT: Monitoring the teaching effectiveness of attending physicians is important to enhancing the quality of graduate medical education. METHODS: We used a critical incident technique with 35 residents representing a cross-section of programmes in a teaching hospital to develop a 23-item rating form. We obtained ratings of 11 attending physicians in internal medicine and general surgery from 54 residents. We performed linear and logistic regression analysis to relate the items on the form to the residents' overall ratings of the attending physicians and the programme directors' ratings of the attending physicians. RESULTS: The residents rated the attending physicians highly in most areas, but lower in provision of feedback, clarity of written communication and cost-effectiveness in making clinical decisions. When we used the residents' overall ratings as the criterion, the most important aspects of attending physicians' teaching were clarity of written communication, cost-effectiveness, commitment of time and energy and whether the resident would refer a family member or friend to the physician. When we used the programme directors' ratings as the criterion, the additional important aspects of performance were concern for the residents' professional well-being, knowledge of the literature and the delivery of clear verbal and written communication. CONCLUSIONS: The critical incident technique can be used to develop an instrument that demonstrates content and construct validity. We found that residents consider commitment of time to teaching and clinical effectiveness to be the most important dimensions of faculty teaching. Other important dimensions include written and verbal communication, cost-effectiveness and concern for residents' professional development.

Assessing physicians' orientation toward lifelong learning.

BACKGROUND: Despite the importance of lifelong learning as an element of professionalism, no psychometrically sound instrument is available for its assessment among physicians. OBJECTIVE: To assess the validity and reliability of an instrument developed to measure physicians' orientation toward lifelong learning. DESIGN: Mail survey. PARTICIPANTS: Seven hundred and twenty-one physicians, of whom 444 (62%) responded. MEASUREMENT: The Jefferson Scale of Physician Lifelong Learning (JSPLL), which includes 19 items answered on a 4-point Likert scale, was used with additional questions about respondents' professional activities related to continuous learning. RESULTS: Factor analysis of the JSPLL yielded 4 subscales entitled: "professional learning beliefs and motivation,""scholarly activities,""attention to learning opportunities," and "technical skills in seeking information," which are consistent with widely recognized features of lifelong learning. The validity of the scale and its subscales was supported by significant correlations with a set of criterion measures that presumably require continuous learning. The internal consistency reliability (coefficient alpha) of the JSPLL was 0.89, and the test-retest reliability was 0.91. CONCLUSIONS: Empirical evidence supports the validity and reliability of the JSPLL.

Systematic review of the literature on assessment, feedback and physicians' clinical performance: BEME Guide No. 7.

BACKGROUND AND CONTEXT: There is a basis for the assumption that feedback can be used to enhance physicians' performance. Nevertheless, the findings of empirical studies of the impact of feedback on clinical performance have been equivocal. OBJECTIVES: To summarize evidence related to the impact of assessment and feedback on physicians' clinical performance. SEARCH STRATEGY: The authors searched the literature from 1966 to 2003 using MEDLINE, HealthSTAR, the Science Citation Index and eight other electronic databases. A total of 3702 citations were identified. INCLUSION AND EXCLUSION CRITERIA: Empirical studies were selected involving the baseline measurement of physicians' performance and follow-up measurement after they received summaries of their performance. DATA EXTRACTION: Data were extracted on research design, sample, dependent and independent variables using a written protocol. DATA SYNTHESIS: A group of 220 studies involving primary data collection was identified. However, only 41 met all selection criteria and evaluated the independent effect of feedback on physician performance. Of these, 32 (74%) demonstrated a positive impact. Feedback was more likely to be effective when provided by an authoritative source over an extended period of time. Another subset of 132 studies examined the effect of feedback combined with other interventions such as educational programmes, practice guidelines and reminders. Of these, 106 studies (77%) demonstrated a positive impact. Two additional subsets of 29 feedback studies involving resident physicians in training and 18 studies examining proxy measures of physician performance across clinical sites or groups of patients were reviewed. The majority of these two subsets also reported that feedback had positive effects on performance. HEADLINE RESULTS: Feedback can change physicians' clinical performance when provided systematically over multiple years by an authoritative, credible source. CONCLUSIONS: The effects of formal assessment and feedback on physician performance are influenced by the source and duration of feedback. Other factors, such as physicians' active involvement in the process, the amount of information reported, the timing and amount of feedback, and other concurrent interventions, such as education, guidelines, reminder systems and incentives, also appear to be important. However, the independent contributions of these interventions have not been well documented in controlled studies. It is recommended that the designers of future theoretical as well as practical studies of feedback separate the effects of feedback from other concurrent interventions.

Clinical vignette-based surveys: a tool for assessing physician practice variation.

Clinical vignette-based surveys have been used for more than 30 years to measure variation in physicians' approaches to the diagnosis and treatment of patients with similar health problems. Vignettes offer advantages over medical record reviews, analysis of claims data, and standardized patients. A vignette-based survey can be completed more quickly than a record review or standardized patient program. Research has shown that vignette-based surveys produce better measures of quality of care than medical record reviews when used to measure differential diagnosis, selection of tests, and treatment decisions. Although standardized patients are preferred when measuring communication and physical examination skills, vignettes are more cost-effective than standardized patients when assessing clinical physicians' decision making. Vignettes offer better opportunities to isolate physicians' decision making and to control case-mix variation than do analyses of claims data sets. Clinical vignette-based surveys are simple and economical tools that can be used to characterize physicians' practice variation.

USMLE Step 2 performance and test administration date in the fourth year of medical school.

PURPOSE: To determine whether the time interval between completing the third-year curriculum and test administration affects a student's USMLE Step 2 score. METHOD: Scores for 846 students in the classes of 2000-2004 were grouped in ten time periods depending on test date. A linear regression model to predict performance on Step 2 using gender, Step 1, and grades in medicine, pediatrics and obstetrics-gynecology was developed based on the class of 1999. Analysis of covariance was used to test the effect of time on scores, adjusting for predicted performance. RESULTS: Step 2 scores decreased significantly (p <.001) across time. Students' mean scores were four points higher than predicted in the early months and five to eight points lower near the end of the senior year. CONCLUSIONS: Students who scheduled Step 2 early in the senior year achieved higher scores, on average, than those who waited until later in the year.

Conceptualization and measurement of clinical competence of residents: a brief rating form and its psychometric properties.

Conceptualization and measurement of clinical competence of residents are of interest to medical educators. Yet there is a scarcity of operational tools with satisfactory psychometric support for measuring clinical competence. In this study, we investigated the underlying structure, criterion-related validity and alpha reliability of a brief rating form (20 items) developed to assess clinical competence of residents. The study sample consisted of 882 physicians (654 men, 228 women) in postgraduate training at Thomas Jefferson University Hospital between 1998 and 2000. Construct validity of the form was supported by factor analysis. Two relevant factors emerged: 'Knowledge, Data-Gathering and Processing Skills', and 'Interpersonal Skills and Attitudes'. Criterion-related validity was supported by significant linear associations between factor scores and performance on the medical licensing examinations. Alpha reliability coefficients for the two factors were 0.98 and 0.97, respectively. This brief rating form can be employed as one measure to evaluate clinical competence of residents with reasonable confidence in its measurement properties.