A multi-site study on medical school selection, performance, motivation and engagement.

Medical schools seek ways to improve their admissions strategies, since the available methods prove to be suboptimal for selecting the best and most motivated students. In this multi-site cross-sectional questionnaire study, we examined the value of (different) selection procedures compared to a weighted lottery procedure, which includes direct admission based on top pre-university grade point averages (≥8 out of 10; top-pu-GPA). We also considered whether students had participated in selection, prior to being admitted through weighted lottery. Year-1 (pre-clinical) and Year-4 (clinical) students completed standard validated questionnaires measuring quality of motivation (Academic Self-regulation Questionnaire), strength of motivation (Strength of Motivation for Medical School-Revised) and engagement (Utrecht Work Engagement Scale-Student). Performance data comprised GPA and course credits in Year-1 and clerkship performance in Year-4. Regression analyses were performed. The response rate was 35% (387 Year-1 and 273 Year-4 students). Top-pu-GPA students outperformed selected students. Selected Year-1 students reported higher strength of motivation than top-pu-GPA students. Selected students did not outperform or show better quality of motivation and engagement than lottery-admitted students. Participation in selection was associated with higher engagement and better clerkship performance in Year-4. GPA, course credits and strength of motivation in Year-1 differed between students admitted through different selection procedures. Top-pu-GPA students perform best in the medical study. The few and small differences found raise questions about the added value of an extensive selection procedure compared to a weighted lottery procedure. Findings have to be interpreted with caution because of a low response rate and small group sizes.

Reflective teaching of medical communication skills with DiViDU: assessing the level of student reflection on recorded consultations with simulated patients.

OBJECTIVE: Acquisition of effective, goal-oriented communication skills requires both practicing skills and reflective thinking. Reflection is a cyclic process of perceiving and analysing communication behaviour in terms of goals and effects and designing improved actions. Based on Korthagen's ALACT reflection model, communication training on history taking was designed. Objectives were to develop rating criteria for assessment of the students' level of reflection and to collect student evaluations of the reflective cycle components in the communication training. METHODS: All second year medical students recorded a consultation with a simulated patient. In DiViDU, a web-based ICT program, students reviewed the video, identified and marked three key events, attached written reflections and provided peer-feedback. Students' written reflections were rated on four reflection categories. A reflection-level score was based on a frequency count of the number of categories used over three reflections. Students filled out an evaluation questionnaire on components of the communication training. RESULTS: Data were analyzed of 304 (90.6%) students. The four reflection categories Observations, Motives, Effects and Goals of behaviour were used in 7-38%. Most students phrased undirected questions for improvement (93%). The average reflection score was 2.1 (S.D. 2.0). All training components were considered instructive. Acting was preferred most. Reviewing video was considered instructive. Self-reflection was considered more difficult than providing written feedback to the reflections of peers. CONCLUSION: Reflection on communication behaviour can be systematically implemented and measured in a structured way. Reflection levels were low, probably indicating a limited notion of goal-oriented attributes of communication skills. PRACTICE IMPLICATIONS: Early introduction of critical self-reflection facilitates acceptance of an important ability for physicians for continued life-long learning and becoming mindful practitioners.

Using standardized video cases for assessment of medical communication skills: reliability of an objective structured video examination by computer.

OBJECTIVE: Using standardized video cases in a computerized objective structured video examination (OSVE) aims to measure cognitive scripts underlying overt communication behavior by questions on knowledge, understanding and performance. In this study the reliability of the OSVE assessment is analyzed using the generalizability theory. METHODS: Third year undergraduate medical students from the Academic Medical Center of the University of Amsterdam answered short-essay questions on three video cases, respectively about history taking, breaking bad news, and decision making. Of 200 participants, 116 completed all three video cases. Students were assessed in three shifts, each using a set of parallel case editions. About half of all available exams were scored independently by two raters using a detailed rating manual derived from the other half. Analyzed were the reliability of the assessment, the inter-rater reliability, and interrelatedness of the three types of video cases and their parallel editions, by computing a generalizability coefficient G. RESULTS: The test score showed a normal distribution. The students performed relatively well on the history taking type of video cases, and relatively poor on decision making and did relatively poor on the understanding ('knows why/when') type of questions. The reliability of the assessment was acceptable (G = 0.66). It can be improved by including up to seven cases in the OSVE. The inter-rater reliability was very good (G = 0.93). The parallel editions of the video cases appeared to be more alike (G = 0.60) than the three case types (G = 0.47). DISCUSSION: The additional value of an OSVE is the differential picture that is obtained about covert cognitive scripts underlying overt communication behavior in different types of consultations, indicated by the differing levels of knowledge, understanding and performance. The validation of the OSVE score requires more research. CONCLUSION AND PRACTICE IMPLICATIONS: A computerized OSVE has been successfully applied with third year undergraduate medical students. The test score meets psychometric criteria, enabling a proper discrimination between adequately and poorly performing students. The high inter-rater reliability indicates that a single rater is permitted.

Summative assessment of medical students' communication skills and professional attitudes through observation in clinical practice.

UNLABELLED: To establish medical students' professional competence for the medical profession, we designed a standardized observation procedure and the Amsterdam Attitude and Communication Scale (AACS) with nine five-point scale items, for summative assessment of their communication skills and professional attitudes. This study examines the reliability of the AACS assessment in clinical practice. In the Academic Medical Centre, Amsterdam, The Netherlands, the performance of 442 fifth year clinical students was judged six times in two settings: behaviour in clinical practice was judged independently twice by a doctor and a nurse; one videotaped patient interview was judged independently by a doctor and by a psychologist. The final mark was obtained by averaging ratings across all six assessments. Raters were 88 doctors, 29 nurses, and three psychologists. MAIN OUTCOME MEASURES: Standard errors (SEs) for absolute judgements indicate measurement precision. Precision of AACS scores is considered sufficient with SEs smaller than 0.25. Multi-disciplinary assessment of students' clinical performance using the AACS is feasible and sufficiently precise (with an overall mean of 3.97 and standard deviation of 0.55, the absolute SE is 0.21). Judgements of behaviour in the clinic were more precise (SEs range from 0.11 to 0.16) than judgements of videotaped interviews (SEs are 0.25 and 0.29). The procedure is sufficiently precise if five or six assessments are combined.

Assessment of medical communication skills by computer: assessment method and student experiences.

BACKGROUND: A computer-assisted assessment (CAA) program for communication skills designated ACT was developed using the objective structured video examination (OSVE) format. This method features assessment of cognitive scripts underlying communication behaviour, a broad range of communication problems covered in 1 assessment, highly standardised assessment and rating procedures, and large group assessments without complex organisation. SETTING: The Academic Medical Centre (AMC) at the University of Amsterdam, the Netherlands. Aims To describe the development of the AMC Communication Test (ACT); to describe our experiences with the examination and rating procedures; to present test score descriptives, and to present the students' opinions of ACT. DESIGN: The ACT presents films on history taking, breaking bad news and shared decision making. Each film is accompanied by 3 types of short essay questions derived from our assessment model: "knows", "knows why/when" and "knows how". Evaluation questions about ACT were integrated into the assessment. Participants A total of 210 third year medical undergraduates were assessed. This study reports on the 110 (53%) students who completed all evaluation questions. RESULTS: Marking 210 examinations took about 17 days. The test score matched a normal distribution and showed a good level of discrimination of the students. About 75% passed the examination. Some support for the validity of our assessment model was found in the students' differential performance on the 3 types of questions. The ACT was well received. Student evaluations confirmed our efforts to develop realistic films that related well to the communication training programme. CONCLUSIONS: The ACT is a useful assessment method which complements interpersonal assessment methods for the evaluation of the medical communication skills of undergraduates.

Cue-responding in a Simulated Bad News Situation: Exploring a Stress Hypothesis.

The stress-coping paradigm of Folkman and Lazarus (1984) was applied to investigate if the communicative reactions of the physician in a bad news transaction are related to the stressfulness of the situation. A standardized video bad news consultation was presented to 88 medical students. To examine their communicative reactions we selected 10 patient cues with different levels of expressed emotion to which the participants responded from the physician's point of view. A strongly positive relationship between expressed emotion and perceived difficulty of the cues and a gender effect occurred, confirming that handling emotions is stressful for physicians. The reluctance of physicians to address the emotionally laden issues of the consultation can be understood as a lack of a frame of reference. The problem-solving strategies, which they apply in the instrumental domain of the consultation, are ineffective when dealing with psychosocial suffering.

Gender differences in fatigue: biopsychosocial factors relating to fatigue in men and women.

BACKGROUND: Fatigue is a common problem, which is found more frequently among women than men. To date, neither the etiology of fatigue nor the factors that explain the gender difference in its incidence are still fully understood. METHODS: In a sample of men (n = 4,681) and women (n = 4,698) (age range, 15-64 years) in the Dutch National Survey of Morbidity and Interventions in General Practice, the gender differences in the underlying biological, psychological, and social factors of fatigue were analyzed. RESULTS: Both general and gender-specific factors were recognized. Men and women who experience complaints of fatigue appeared to be younger and more highly educated. They had more acute health complaints and more psychosocial problems and also showed a lower level of perceived health. Among women, only gender-specific biological complaints and psychosocial problems were related to fatigue. In addition, relevant sociodemographic variables included taking care of young children and being employed. Among men, fatigue was particularly related to having handicaps and severe chronic complaints. Taking care of young children did not make a difference in the male sample. CONCLUSIONS: Fatigue can only be adequately understood in a multicausal model with biomedical and psychosocial factors. Complaints of fatigue are too often ignored in general practice. By adopting a patient-centered style of communication, physicians can acquire a more complete picture of the patients' fatigue.

Teaching clinically experienced physicians communication skills. A review of evaluation studies.

CONTEXT: Interest in the teaching of communication skills in medical schools has increased since the early seventies but, despite this growing interest, relatively limited curricular time is spent on the teaching of communication skills. The limited attention to the teaching of these skills applies even more to the physicians' clinical years, when attention becomes highly focused on biomedical and technical competence. Continuing training after medical school is necessary to refresh knowledge and skills, to prohibit decline of performance and to establish further improvements. OBJECTIVE: This review provides an overview of evaluation studies of communication skills training programmes for clinically experienced physicians who have finished their undergraduate medical education. The review focuses on the training objectives, the applied educational methods, the evaluation methodology and instruments, and training results. METHODS: CD-ROM searches were performed on MedLine and Psychlit, with a focus on effect-studies dating from 1985. RESULTS: Fifteen papers on 14 evaluation studies were located. There appears to be some consistency in the aims and methods of the training programmes. Course effect measurements include physician self-ratings, independent behavioural observations and patient outcomes. Most of the studies used inadequate research designs. Overall, positive training effects on the physicians' communication behaviour are found on half or less of the observed behaviours. Studies with the most adequate designs report the fewest positive training effects. CONCLUSION: Several reasons are discussed to explain the limited findings. Future research may benefit from research methods which focus on factors that inhibit and facilitate the physicians' implementation of skills into actual behaviours in daily practice.

INTERACT-CANCER. The development and evaluation of a computer-assisted course on communication skills for medical specialists in oncology.

One of the main problems in training medical specialists in communication skills is the fact that very few specialists participate in such courses. Most courses on communication skills are hard to combine with a busy medical practice because of the course attendance requirement on working days. Interact-Cancer has been developed to facilitate training in communication skills for medical specialists. It is an interactive, computer-assisted course, which fits relatively easily into the busy schedule of medical specialists in oncology because of its flexible character. The course can be followed individually by the participant in his/her own department at suitable periods of time. Interact-Cancer has introduced a new training method. In this study, the appeal of the method to the target group has been tested. Thirty-three medical specialists participated in Interact-Cancer. They evaluated the course as worthwhile, relevant to daily oncological practice, attractive and pleasant. They stated that the course brought inadequacies in their own behaviour to their attention and encouraged them to change their communicative behaviour.