Effectiveness of a supervisor training on quality of feedback to internal medicine residents: a controlled longitudinal multicentre study.

OBJECTIVES: High-quality feedback on different dimensions of competence is important for resident learning. Supervisors may need additional training and information to fulfil this demanding task. This study aimed to evaluate whether a short and simple training improves the quality of feedback residents receive from their clinical supervisors in daily practice. DESIGN: Longitudinal quasi-experimental controlled study with a pretest/post-test design. We collected multiple premeasurements and postmeasurements for each supervisor over 2 years. A repeated measurements ANOVA was performed on the data. SETTING: Internal medicine departments of seven Dutch teaching hospitals. PARTICIPANTS: Internal medicine supervisors (n=181) and residents (n=192). INTERVENTION: Half of the supervisors attended a short 2.5-hour training session during which they could practise giving feedback in a simulated setting using video fragments. Highly experienced internal medicine educators guided the group discussions about the feedback. The other half of the supervisors formed the control group and received no feedback training. OUTCOME MEASURES: Residents rated the quality of supervisors' oral feedback with a previously validated questionnaire. Furthermore, the completeness of the supervisors' written feedback on evaluation forms was analysed. RESULTS: The data showed a significant increase in the quality of feedback after the training F (1, 87)=6.76, p=0.04. This effect remained significant up to 6 months after the training session. CONCLUSIONS: A short training session in which supervisors practise giving feedback in a simulated setting increases the quality of their feedback. This is a promising outcome since it is a feasible approach to faculty development.

Training medical students for the twenty-first century: Rationale and development of the Utrecht curriculum "CRU+".

AIM: The aim of this report, written for the 40th anniversary issue of Medical Teacher, is to document 20 years of development of the Utrecht undergraduate medical curriculum, as both to exhibit accountability and to inform the community of the process and choices that can be made in long-term curriculum development. METHODS: We used the SPICES model, created by Medical Teacher's Editor Ronald Harden and colleagues in 1984. RESULTS: The Utrecht six-year program, now called "CRU+", has many distinct features that were introduced, most of which are well documented. A limited selection includes • A new 3+3 years Bachelor-Master structure following the EU Bologna rules leading to MD registration for cohorts of about 300. • Horizontally integrated classroom teaching of basic sciences with clinical disciplines predominantly in groups of 12 and limited lectures. • Mandatory knowledge retention tests, retesting the clinically relevant core knowledge from block tests of semesters one through four. • Vertical integration not only linking clinical experience with background knowledge, but also exemplified by a stepwise increase in health care responsibilities throughout the curriculum. • A final year focussing on growth towards the level of a primary responsible physician in a 12-week sub-internship for a limited number of patients and beds, in a chosen specialty. The student is called a semi-physician in the clerkship of this transitional year to residency. • Teaching skills training for all medical graduates, an elective teaching rotation and various peer-teaching arrangements throughout the curriculum. • Integrated semi-longitudinal clerkships with an assessment focus on entrustment decisions for Entrustable Professional Activities. CONCLUSION: UMC Utrecht has made a continuous attempt to both develop its medical curriculum and to study and report on its development in the literature, regarding new methods found and insights derived. UMC Utrecht will remain committed to developing training to meet twenty-first century demands of medical graduates.

Physician job satisfaction related to actual and preferred job size.

BACKGROUND: Job satisfaction is essential for physicians' well-being and patient care. The work ethic of long days and hard work that has been advocated for decades is acknowledged as a threat for physicians' job satisfaction, well-being, and patient safety. Our aim was to determine the actual and preferred job size of physicians and to investigate how these and the differences between them influence physicians' job satisfaction. METHOD: Data were retrieved from a larger, longitudinal study among physicians starting medical training at Groningen University in 1982/83/92/93 (N = 597). Data from 506 participants (85%) were available for this study. We used regression analysis to investigate the influence of job size on physicians' job satisfaction (13 aspects) and ANOVA to examine differences in job satisfaction between physicians wishing to retain, reduce or increase job size. RESULTS: The majority of the respondents (57%) had an actual job size less than 1.0 FTE. More than 80% of all respondents preferred not to work full-time in the future. Respondents' average actual and preferred job sizes were .85 FTE and .81 FTE, respectively. On average, respondents who wished to work less (35% of respondents) preferred a job size reduction of 0.18 FTE and those who wished to work more (12%) preferred an increase in job size of 0.16 FTE. Job size influenced satisfaction with balance work-private hours most (ß = -.351). Physicians who preferred larger job sizes were - compared to the other groups of physicians - least satisfied with professional accomplishments. CONCLUSIONS: A considerable group of physicians reported a gap between actual and preferred job size. Realizing physicians' preferences as to job size will hardly affect total workforce, but may greatly benefit individual physicians as well as their patients and society. Therefore, it seems time for a shift in work ethic.

Integrated and implicit: how residents learn CanMEDS roles by participating in practice.

CONTEXT: Learning outcomes for residency training are defined in competency frameworks such as the CanMEDS framework, which ultimately aim to better prepare residents for their future tasks. Although residents' training relies heavily on learning through participation in the workplace under the supervision of a specialist, it remains unclear how the CanMEDS framework informs practice-based learning and daily interactions between residents and supervisors. OBJECTIVES: This study aimed to explore how the CanMEDS framework informs residents' practice-based training and interactions with supervisors. METHODS: Constructivist grounded theory guided iterative data collection and analyses. Data were collected by direct observations of residents and supervisors, combined with formal and field interviews. We progressively arrived at an explanatory theory by coding and interpreting the data, building provisional theories and through continuous conversations. Data analysis drew on sensitising insights from communities of practice theory, which provided this study with a social learning perspective. RESULTS: CanMEDS roles occurred in an integrated fashion and usually remained implicit during interactions. The language of CanMEDS was not adopted in clinical practice, which seemed to impede explicit learning interactions. The CanMEDS framework seemed only one of many factors of influence in practice-based training: patient records and other documents were highly influential in daily activities and did not always correspond with CanMEDS roles. Additionally, the position of residents seemed too peripheral to allow them to learn certain aspects of the Health Advocate and Leader roles. CONCLUSIONS: The CanMEDS framework did not really guide supervisors' and residents' practice or interactions. It was not explicitly used as a common language in which to talk about resident performance and roles. Therefore, the extent to which CanMEDS actually helps improve residents' learning trajectories and conversations between residents and supervisors about residents' progress remains questionable. This study highlights the fact that the reification of competency frameworks into the complexity of practice-based learning is not a straightforward exercise.

Participation and selection effects of a voluntary selection process.

Many different medical school selection processes are used worldwide. In this paper, we examine the effect of (1) participation, and (2) selection in a voluntary selection process on study performance. We included data from two cohorts of medical students admitted to Erasmus MC, Rotterdam and VUmc, Amsterdam, The Netherlands and compared them to previously published data from Groningen medical school, The Netherlands. All included students were admitted based on either (1) a top pre-university grade point average, or (2) a voluntary selection process, or (3) weighted lottery. We distinguished between lottery-admitted students who had participated in the voluntary selection process and had been rejected, and lottery-admitted students who had not participated. Knowledge test scores, study progress, and professionalism scores were examined using ANCOVA modelling, logistic regression, and Bonferroni post hoc multiple-comparison tests, controlling for gender and cohort. For written test grades, results showed a participation effect at Groningen medical school and Erasmus MC (p < 0.001), and a selection effect at VUmc (p < 0.05). For obtained course credits, results showed a participation effect at all universities (p < 0.01) and a selection effect at Groningen medical school (p < 0.005). At Groningen medical school, a participation effect seemed apparent in on time first-year completion (p < 0.05). Earlier reported selection and participation effects in professionalism scores at Groningen medical school were not apparent at VUmc. Top pre-university students performed well on all outcome measures. For both the participation effect and the selection effect, results differed between universities. Institutional differences in curricula and in the design of the selection process seem to mediate relations between the different admissions processes and performance. Further research is needed for a deeper understanding of the influence of institutional differences on selection outcomes.

Impact of vocational interests, previous academic experience, gender and age on Situational Judgement Test performance.

Situational Judgement Tests (SJTs) are increasingly implemented in medical school admissions. In this paper, we investigate the effects of vocational interests, previous academic experience, gender and age on SJT performance. The SJT was part of the selection process for the Bachelor's degree programme in Medicine at University of Groningen, the Netherlands. All applicants for the academic year 2015-2016 were included and had to choose between learning communities Global Health (n = 126), Sustainable Care (n = 149), Intramural Care (n = 225), or Molecular Medicine (n = 116). This choice was used as a proxy for vocational interest. In addition, all graduate-entry applicants for academic year 2015-2016 (n = 213) were included to examine the effect of previous academic experience on performance. We used MANCOVA analyses with Bonferroni post hoc multiple comparisons tests for applicant performance on a six-scenario SJT. The MANCOVA analyses showed that for all scenarios, the independent variables were significantly related to performance (Pillai's Trace: 0.02-0.47, p < .01). Vocational interest was related to performance on three scenarios (p < .01). Graduate-entry applicants outperformed all other groups on three scenarios (p < .01) and at least one other group on the other three scenarios (p < .01). Female applicants outperformed male applicants on three scenarios (p < .01) and age was positively related to performance on two scenarios (p < .05). A good fit between applicants' vocational interests and SJT scenario was related to better performance, as was previous academic experience. Gender and age were related to performance on SJT scenarios in different settings. Especially the first effect might be helpful in selecting appropriate candidates for areas of health care in which more professionals are needed.

A feedback system in residency to evaluate CanMEDS roles and provide high-quality feedback: Exploring its application.

INTRODUCTION: Residents benefit from regular, high quality feedback on all CanMEDS roles during their training. However, feedback mostly concerns Medical Expert, leaving the other roles behind. A feedback system was developed to guide supervisors in providing feedback on CanMEDS roles. We analyzed whether feedback was provided on the intended roles and explored differences in quality of written feedback. METHODS: In the feedback system, CanMEDS roles were assigned to five authentic situations: Patient Encounter, Morning Report, On-call, CAT, and Oral Presentation. Quality of feedback was operationalized as specificity and inclusion of strengths and improvement points. Differences in specificity between roles were tested with Mann-Whitney U tests with a Bonferroni correction (α = 0.003). RESULTS: Supervisors (n = 126) provided residents (n = 120) with feedback (591 times). Feedback was provided on the intended roles, most frequently on Scholar (78%) and Communicator (71%); least on Manager (47%), and Collaborator (56%). Strengths (78%) were mentioned more frequently than improvement points (52%), which were lacking in 40% of the feedback on Manager, Professional, and Collaborator. Feedback on Scholar was more frequently (p = 0.000) and on Reflective Professional was less frequently (p = 0.003) specific. DISCUSSION AND CONCLUSION: Assigning roles to authentic situations guides supervisors in providing feedback on different CanMEDS roles. We recommend additional supervisor training on how to observe and evaluate the roles.

What supervisors say in their feedback: construction of CanMEDS roles in workplace settings.

The CanMEDS framework has been widely adopted in residency education and feedback processes are guided by it. It is, however, only one of many influences on what is actually discussed in feedback. The sociohistorical culture of medicine and individual supervisors' contexts, experiences and beliefs are also influential. Our aim was to find how CanMEDS roles are constructed in feedback in a postgraduate curriculum-in-action. We applied a set of discourse analytic tools to written feedback from 591 feedback forms from 7 hospitals, including 3150 feedback comments in which 126 supervisors provided feedback to 120 residents after observing their performance in authentic settings. The role of Collaborator was constructed in two different ways: a cooperative discourse of equality with other workers and patients; and a discourse, which gave residents positions of power-delegating, asserting and 'taking a firm stance'. Efficiency-being fast and to the point emerged as an important attribute of physicians. Patients were seldom part of the discourses and, when they were, they were constructed as objects of communication and collaboration rather than partners. Although some of the discourses are in line with what might be expected, others were in striking contrast to the spirit of CanMEDS. This study's findings suggest that it takes more than a competency framework, evaluation instruments, and supervisor training to change the culture of workplaces. The impact on residents of training in such demanding, efficiency-focused clinical environments is an important topic for future research.

Vertically integrated medical education and the readiness for practice of graduates.

BACKGROUND: Medical curricula become more and more vertically integrated (VI) to prepare graduates better for clinical practice. VI curricula show early clinical education, integration of biomedical sciences and focus on increasing clinical responsibility levels for trainees. Results of earlier questionnaire-based studies indicate that the type of the curriculum can affect the perceived preparedness for work as perceived by students or supervisors. The aim of the present study is to determine difference in actual performance of graduates from VI and non-VI curricula. METHODS: We developed and implemented an authentic performance assessment based on different facets of competence for medical near-graduates in the role of beginning residents on a very busy day. Fifty nine candidates participated: 30 VI (Utrecht, The Netherlands) and 29 non-VI (Hamburg, Germany). Two physicians, one nurse and five standardized patients independently assessed each candidate on different facets of competence. Afterwards, the physicians indicated how much supervision they estimated each candidate would require on nine so called "Entrustable Professional Activities (EPAs)" unrelated to the observed scenarios. RESULTS: Graduates from a VI curriculum received significantly higher scores by the physicians for the facet of competence "active professional development", with features like 'reflection' and 'asking for feedback'. In addition, VI graduates scored better on the EPA "solving a management problem", while the non-VI graduates got higher scores for the EPA "breaking bad news". CONCLUSIONS: This study gives an impression of the actual performance of medical graduates from VI and non-VI curricula. Even though not many differences were found, VI graduates got higher scores for features of professional development, which is important for postgraduate training and continuing education.

The transferability of Western concepts to other cultures: validation of the Zuckerman-Kuhlman Personality Questionnaire in a Saudi Arabic context.

BACKGROUND: The importance of medical education research in Saudi Arabia has been acknowledged increasingly and a lot of concepts used have been derived from the Western world. The question arises, however, whether Western concepts and questionnaires are transferable to societies with different cultures. The aim of this study was to investigate the instrument structure and the reliability of the Arabic version of the Zuckerman-Kuhlman Personality Questionnaire-medium (ZKPQ-m). METHODS: Three statistical methods with decreased amount of strictness were used to analyse our data: Confirmatory Factor Analysis, Procrustes rotation and Principal Component Analysis. RESULTS: Our outcomes did not confirm the original instrument structure. Instead, we found four interpretable components: Emotional Instability, Impulse-seeking, Activeness and Self-Control. However, the amount of explained variance was not very high and the internal consistencies ranged from unsatisfactory to only moderate. The data showed a high percentage of respondents agreeing with more than three items of the Infrequency scale, which may be attributable to the collectivistic culture in Saudi Arabia. CONCLUSION: We did not succeed in replicating the ZKPQ structure in the Arabic context. Social desirability, a common characteristic in collectivistic cultures, may have reduced the replicability of the internal structure of the ZKPQ-m. Different methods to measure concepts in collectivistic cultures may help to get round social desirability.

Selection and study performance: comparing three admission processes within one medical school.

OBJECTIVES: This study was conducted to: (i) analyse whether students admitted to one medical school based on top pre-university grades, a voluntary multifaceted selection process, or lottery, respectively, differed in study performance; (ii) examine whether students who were accepted in the multifaceted selection process outperformed their rejected peers, and (iii) analyse whether participation in the multifaceted selection procedure was related to performance. METHODS: We examined knowledge test and professionalism scores, study progress and dropout in three cohorts of medical students admitted to the University of Groningen, the Netherlands in 2009, 2010 and 2011 (n = 1055). We divided the lottery-admitted group into, respectively, students who had not participated and students who had been rejected in the multifaceted selection process. We used ancova modelling, logistic regression and Bonferroni post hoc multiple-comparison tests and controlled for gender and cohort. RESULTS: The top pre-university grade group achieved higher knowledge test scores and more Year 1 course credits than all other groups (p < 0.05). This group received the highest possible professionalism score more often than the lottery-admitted group that had not participated in the multifaceted selection process (p < 0.05). The group of students accepted in the multifaceted selection process obtained higher written test scores than the lottery-admitted group that had not participated (p < 0.05) and achieved the highest possible professionalism score more often than both lottery-admitted groups. The lottery-admitted group that had not participated in the multifaceted selection process earned fewer Year 1 and 2 course credits than all other groups (p < 0.05). Dropout rates differed among the groups (p < 0.05), but correction for multiple comparisons rendered all pairwise differences non-significant. CONCLUSIONS: A top pre-university grade point average was the best predictor of performance. For so-called non-academic performance, the multifaceted selection process was efficient in identifying applicants with suitable skills. Participation in the multifaceted selection procedure seems to be predictive of higher performance. Further research is needed to assess whether our results are generalisable to other medical schools.

The influence of achievement before, during and after medical school on physician job satisfaction.

In this longitudinal study, we investigated the relationship between physicians' prior achievements (before, during and after medical school) and job satisfaction, and tested the two lines of reasoning that prior achievements influence job satisfaction positively or negatively, respectively. The participants were graduates who started their medical training in 1982 (n = 147), 1983 (n = 154), 1992 (n = 143) and 1993 (n = 153). We operationalised job satisfaction as satisfaction (on a 10-point scale) with 13 cognitive, affective and instrumental aspects of the participants' jobs. The measures of achievement before, during and after medical school included pre-university grade point average, study progress and a residency position in the specialty of first choice, respectively. We included the effect of curriculum type (problem-based learning versus traditional), gender and years of experience as moderator variables. Higher achievers before and during medical school were more satisfied about their income (ß = .152, p < .01 and ß = .149, p < .05), but less satisfied with their opportunities for personal development (ß = -.159, p < .05). High achievers after medical school were more satisfied with professional accomplishments (ß = .095, p < .05), with appreciation from support personnel (ß = .154, p < .01) and from patients (ß = .120, p < .05). Effect sizes were small. Prior achievements influenced job satisfaction. The direction of the influences depended on the job satisfaction aspect in question, which indicates that it is important to distinguish between aspects of job satisfaction. To optimize job satisfaction of high achievers, it is important for graduates to obtain their preferred specialty. Furthermore, it is vital to provide them with enough opportunities for further development.

Newly qualified doctors' views on the significance and accessibility of career advice during medical training in Saudi Arabia.

BACKGROUND: Career advice is an important instrument to help students with the proper specialty selection. The study aims (1) to explore the views of newly graduated doctors in Saudi Arabia about their experience with the current status of career support system during medical training and (2) to identify cross-cultural similarities and differences. METHODS: A cross-sectional design study was conducted using a questionnaire to elicit the responses of participants from newly qualified doctors concerning the availability and significance of career advice. SPSS (version 11.0; Chicago, IL) was used to analyze the data and statistical tests, such as chi-square and unpaired t tests, were used to analyze the observations. RESULTS: A response rate of 94.7% was obtained. Among this group, 102 were males and 78 were females. Only 53% did receive career advice. The majority of men felt that career advice during medical studies was inadequate, while women were less negative (69% versus 32%; p = 0.0001). Furthermore, men were more disappointed about the possibilities for career advice after graduating than women (34% versus 13%, p = 0.0001). CONCLUSIONS: The results show that only half of newly graduated doctors had received any career advice during medical training. As the health care system cannot afford the potential waste of time and resources for doctors, career guidance should begin in undergraduate training so that the process of thinking about their future career starts longtime before they make their career choice.

Personality types and specialist choices in medical students.

BACKGROUND: Research on the correlation between personality and students' specialty choice is helpful in their career counselling process and in predicting the future distribution of the specialties in a country. AIMS: This study is the first of its kind in the Arab world. The research questions were: (1) What is the influence of gender on the personality profiles of medical students? (2) What are the personality profiles of students categorized according to their preferred specialist choices? (3) What are the preferred career choices of students categorized according to the stage of their medical education? METHOD: A cross-sectional study was performed at King Khalid University Medical School including 590 students during the academic year 2010-2011. A long version of the Zuckerman-Kuhlman personality questionnaire measuring five personality factors was used. Students were also asked for their specialty interests. Students were asked by means of a written questionnaire. RESULTS: Study response was 92.5%. Surgery was the single most popular specialty amongst both male and female students. Males had significantly higher scores on the 'impulsive sensation seeking' scale and students preferring a surgery specialty had the highest score on the 'impulsive sensation seeking', 'neuroticism-anxiety', 'aggression-hostility' and 'sociability' scales. Hospital-based, surgical and primary care specialties became more popular as students progressed through their undergraduate years. CONCLUSIONS: Different personality types have distinct preferences in medical students' choice of careers. Personality and specialty choice research can enhance career counselling of medical students and fresh graduates. This also has implications for predicting the specialty distribution of the future health careers.

What are we preparing them for? Development of an inventory of tasks for medical, surgical and supportive specialties.

BACKGROUND: Internationally, postgraduate medical education (PGME) has shifted to competency-based training. To evaluate the effects of this shift on the outcomes of PGME appropriate instruments are needed. AIM: To provide an inventory of tasks specialists perform in practice, which can be used as an instrument to evaluate the outcomes of PGME across disciplines. METHODS: Following methodology from job analysis in human resource management, we used document analyses, observations, interviews and questionnaires. Two thousand seven hundred and twenty eight specialists were then asked to indicate how frequently they performed each task in the inventory, and to suggest additional tasks. Face and content validity was evaluated using interviews and the questionnaire. Tasks with similar content were combined in a total of 12 clusters. Internal consistency was evaluated by calculating Cronbach's alpha. Construct validity was determined by examining predefined differences in task performance between medical, surgical and supportive disciplines. RESULTS: Seven hundred and six specialists (36%) returned the questionnaire. The resulting inventory of 91 tasks showed adequate face and content validity. Internal consistency of clusters of tasks was adequate. Significant differences in task performance between medical, surgical and supportive disciplines indicated construct validity. CONCLUSION: We established a comprehensive, generic and valid inventory of tasks of specialists which appears to be applicable across medical, surgical and supportive disciplines.

Specialty preferences: trends and perceptions among Saudi undergraduate medical students.

BACKGROUND: The exploration of specialty choices by medical students is a hot debate as it affects several important determinants of health care delivery. This study was carried out to determine variation in specialty preferences during medical school training and the perceptions that affect students' specialty choice. METHODS: A cross-sectional questionnaire-based study was performed on 590 students with a 93.22% response rate and covered queries on demography, specialty choices, and perceptions influencing specialty choices. Class-wise analysis of specialty choices was carried out. RESULTS: The most preferred specialty expressed by male students was surgery, followed by internal medicine and orthopedics, while most preferred by female students were surgery, followed by pediatrics and ophthalmology. Male students' emphasized factors like less competitive field, shortage of specialists, and diversity of patients while the prestige of specialty and teaching opportunities had a greater impact on female students. CONCLUSIONS: Surgery, internal medicine, pediatrics, orthopedics, and ophthalmology were the most preferred specialty choices. Gender preference was observed to affect choices of few specialties such as orthopedics and obstetrics/gynecology. Perceptions which have an impact on specialty selection of male and female students may reflect a different tempo of growing up in men and women.