Assessing clinical reasoning in undergraduate medical students during history taking with an empirically derived scale for clinical reasoning indicators.

BACKGROUND: The clinical reasoning process, which requires biomedical knowledge, knowledge about problem-solving strategies, and knowledge about reasons for diagnostic procedures, is a key element of physicians' daily practice but difficult to assess. The aim of this study was to empirically develop a Clinical Reasoning Indicators-History Taking-Scale (CRI-HT-S) and to assess the clinical reasoning ability of advanced medical students during a simulation involving history taking. METHODS: The Clinical Reasoning Indictors-History Taking-Scale (CRI-HT-S) including a 5-point Likert scale for assessment was designed from clinical reasoning indicators identified in a qualitative study in 2017. To assess indicators of clinical reasoning ability, 65 advanced medical students (semester 10, n = 25 versus final year, n = 40) from three medical schools participated in a 360-degree competence assessment in the role of beginning residents during a simulated first workday in hospital. This assessment included a consultation hour with five simulated patients which was videotaped. Videos of 325 patient consultations were assessed using the CRI-HT-S. A factor analysis was conducted and the students' results were compared according to their advancement in undergraduate medical training. RESULTS: The clinical reasoning indicators of the CRI-HT-S loaded on three factors relevant for clinical reasoning: 1) focusing questions, 2) creating context, and 3) securing information. Students reached significantly different scores (p < .001) for the three factors (factor 1: 4.07 ± .47, factor 2: 3.72 ± .43, factor 3: 2.79 ± .83). Students in semester 10 reached significantly lower scores for factor 3 than students in their final year (p < .05). CONCLUSIONS: The newly developed CRI-HT-S worked well for quantitative assessment of clinical reasoning indicators during history taking. Its three-factored structure helped to explore different aspects of clinical reasoning. Whether the CRI-HT-S has the potential to be used as a scale in objective structured clinical examinations (OCSEs) or in workplace-based assessments of clinical reasoning has to be investigated in further studies with larger student cohorts.

Entrustable professional activities and facets of competence in a simulated workplace-based assessment for advanced medical students.

Background: Competence-based assessment formats in medical education usually focus on individual facets of competence (FOCs). The concept of 'Entrustable Professional Activities' (EPAs) encompasses supervisors' decisions on which level of supervision a trainee requires to perform a professional activity including several FOCs. How the different FOCs as perceived by clinician raters contribute to entrustment decisions is yet unclear.Objective: How do FOC perceptions relate to entrustment-decisions?Methods: Sixty-seven advanced medical students participated in an assessment simulating the first day of a resident physician. Participants were rated by supervisors for seven FOCs and twelve EPAs.Results: There was a positive correlation between FOC and EPA scores. Each EPA displayed a different correlation pattern with FOC ratings.Discussion: For most EPAs high levels of entrustment were associated with high ratings for selected FOCs. The results are in alignment with the assumption that each EPA encompasses a different set of FOCs.Conclusions: In our simulated workplace-based assessment, entrustment decisions for EPAs reflect the FOCs observed in a trainee. Thus, assessment of FOCs alongside with EPA ratings could add to the understanding of factors contributing to entrustment decisions.

Validation of a competence-based assessment of medical students' performance in the physician's role.

BACKGROUND: Assessing competence of advanced undergraduate medical students based on performance in the clinical context is the ultimate, yet challenging goal for medical educators to provide constructive alignment between undergraduate medical training and professional work of physicians. Therefore, we designed and validated a performance-based 360-degree assessment for competences of advanced undergraduate medical students. METHODS: This study was conducted in three steps: 1) Ten facets of competence considered to be most important for beginning residents were determined by a ranking study with 102 internists and 100 surgeons. 2) Based on these facets of competence we developed a 360-degree assessment simulating a first day of residency. Advanced undergraduate medical students (year 5 and 6) participated in the physician's role. Additionally knowledge was assessed by a multiple-choice test. The assessment was performed twice (t1 and t2) and included three phases: a consultation hour, a patient management phase, and a patient handover. Sixty-seven (t1) and eighty-nine (t2) undergraduate medical students participated. 3) The participants completed the Group Assessment of Performance (GAP)-test for flight school applicants to assess medical students' facets of competence in a non-medical context for validation purposes. We aimed to provide a validity argument for our newly designed assessment based on Messick's six aspects of validation: (1) content validity, (2) substantive/cognitive validity, (3) structural validity, (4) generalizability, (5) external validity, and (6) consequential validity. RESULTS: Our assessment proved to be well operationalised to enable undergraduate medical students to show their competences in performance on the higher levels of Bloom's taxonomy. Its generalisability was underscored by its authenticity in respect of workplace reality and its underlying facets of competence relevant for beginning residents. The moderate concordance with facets of competence of the validated GAP-test provides arguments of convergent validity for our assessment. Since five aspects of Messick's validation approach could be defended, our competence-based 360-degree assessment format shows good arguments for its validity. CONCLUSION: According to these validation arguments, our assessment instrument seems to be a good option to assess competence in advanced undergraduate medical students in a summative or formative way. Developments towards assessment of postgraduate medical trainees should be explored.

Medical knowledge and teamwork predict the quality of case summary statements as an indicator of clinical reasoning in undergraduate medical students.

Background: Clinical reasoning refers to a thinking process including medical problem solving and medical decision making skills. Several studies have shown that the clinical reasoning process can be influenced by a number of factors, e.g. context or personality traits, and the results of this thinking process are expressed in case presentation. The aim of this study was to identify factors, which predict the quality of case summary statements as an indicator of clinical reasoning of undergraduate medical students in an assessment simulating the first day of residency. Methods: To investigate factors predicting aspects of clinical reasoning 67 advanced undergraduate medical students participated in the role of a beginning resident in our competence-based assessment, which included a consultation hour, a patient management phase, and a handover. Participants filled out a Post Encounter Form (PEF) to document their case summary statements and other aspects of clinical reasoning. After each phase, they filled out the Strain Perception Questionnaire (STRAIPER) to measure their situation dependent mental strain. To assess medical knowledge the participants completed a 100 questions multiple choice test. To measure stress resistance, adherence to procedures, and teamwork students took part in the Group Assessment of Performance (GAP) test for flight school applicants. These factors were included in a multiple linear regression analysis. Results: Medical knowledge and teamwork predicted the quality of case summary statements as an indicator of clinical reasoning of undergraduate medical students and explained approximately 20.3% of the variance. Neither age, gender, undergraduate curriculum, academic advancement nor high school grade point average of the medical students of our sample had an effect on their clinical reasoning skills. Conclusion: The quality of case summary statements as an indicator of clinical reasoning can be predicted in undergraduate medical students by their medical knowledge and teamwork. Students should be supported in developing abilities to work in a team and to acquire long term knowledge for good case summary statements as an important aspect of clinical reasoning.

Interprofessional assessment of medical students' competences with an instrument suitable for physicians and nurses.

BACKGROUND: Physicians need a set of specific competences to perform well in interprofessional teams in their first year of residency. These competences should be achieved with graduation from medical school. Assessments during undergraduate medical studies are mostly rated by supervisors only. The aim of our study was to compare the rating of core facets of competence of medical students late in their undergraduate training as well as the rating confidence between three different groups of assessors (supervisors, residents, and nurses) in an assessment simulating the first day of residency. METHODS: Sixty-seven advanced medical students from three different medical schools (Hamburg, Oldenburg and Munich) participated in a 360-degree assessment simulating the first working day of a resident. Each participant was rated by three assessors - a supervisor, a resident and a nurse - in seven facets of competence relevant for the first year of residency: (1) responsibility, (2) teamwork and collegiality, (3) knowing and maintaining own personal bounds and possibilities, (4) structure, work planning and priorities, (5) coping with mistakes, (6) scientifically and empirically grounded method of working, and (7) verbal communication with colleagues and supervisors. Means of all assessed competences and confidences of judgement of the three rating groups were compared. Additionally, correlations between assessed competences and confidence of judgement within each group of raters were computed. RESULTS: All rating groups showed consistent assessment decisions (Cronbach's α: supervisors = .90, residents = .80, nurses = .78). Nurses assessed the participants significantly higher in all competences compared to supervisors and residents (all p ≤ .05) with moderate and high effect sizes (d = .667-1.068). While supervisors' and residents' ratings were highest for "teamwork and collegiality", participants received the highest rating by nurses for "responsibility". Competences assessed by nurses were strongly positively correlated with their confidence of judgment while supervisors' assessments correlated only moderately with their confidence of judgment in two competences. CONCLUSIONS: Different professional perspectives provide differentiated competence ratings for medical students in the role of a beginning resident. Rating confidence should be enhanced by empirically derived behavior checklists with anchors, which need to be included in rater training to decrease raters' subjectivity.

Perceived strain of undergraduate medical students during a simulated first day of residency.

BACKGROUND: Residents face demanding situations on the job and have been found to perceive high levels of strain. Medical students also reported a high degree of strain and even depressive tendencies when entering their clinical rotations. The aim of this study was to explore the perceived strain of medical students from different undergraduate curricula and at different stages of academic advancement during different phases of an assessment simulating a resident's first day in hospital. METHODS: Sixty-seven undergraduate medical students participated in the following three phases of the assessment in the role of a resident: a consultation hour with five simulated patients, a management phase with interprofessional contact, and a patient handover with a colleague. They completed the Strain Perception Questionnaire (STRAIPER) after each phase. Students from different undergraduate curricula (VI: vertically integrated, n = 35 versus non-VI: not vertically integrated, n = 26) and different academic advancement (semester 10, n = 26 versus final year, n = 41) were compared. RESULTS: All students showed the highest strain level after the management phase compared to the consultation hour and the handover. Medical students from a non-VI curriculum felt significantly more strain in the dimension of agitation (p < .05) after the consultation hour compared to students from a VI curriculum and compared to the management phase and the handover. No significant difference in perceived strain was found between students from semester 10 compared to final year students. CONCLUSIONS: During the consultation hour and the handover with a colleague medical students faced tasks which are familiar to them from undergraduate education. Their higher strain levels during the management phase might occur because they are confronted with unfamiliar tasks and decisions. Feeling responsible for the right actions in this phase of multitasking and professional interaction might have added to the strain students perceived during this phase. Patient management should be emphasized more in any type of undergraduate medical curriculum.

Differences between medical student and faculty perceptions of the competencies needed for the first year of residency.

BACKGROUND: Different guidelines and frameworks like the CanMEDs model or entrustable professional activities (EPAs) describe competencies required for successful and professional work of residents. Not all competencies are of equal importance for graduates when they start their residency. The aim of this study was to evaluate the relevance of different competencies for a first year resident from the perspective of physicians and medical students. METHODS: In an online study, 178 of 475 surgeons and internists including residents and attendings and 102 of 728 first and last year undergraduate medical students from the University Medical Center Hamburg-Eppendorf ranked 25 competencies according to their relevance for entrustment decisions in first year residents. The rankings of the competencies by residents and attendings and by first year and last year medical student were compared. Additionally, the rankings were also compared to the literature. RESULTS: Physicians and medical students rated 'Responsibility' as the most important competency for first year residents. Physicians ranked 'Teamwork and collegiality' and 'Structure, work planning and priorities' within the top 10 competencies significantly higher than medical students. The competency ranks between attendings and residents only showed one significant difference between attendings and residents, where 'Coping with mistakes', was ranked significantly higher by residents. Medical students ranked 'Active listening to patients', 'Advising patients' and 'Handling emotions of patients and their relatives' significantly higher than physicians. Final year students ranked 'Structure, work planning and priorities', 'Coping with mistakes', and 'Verbal communication with colleagues and supervisors' significantly higher than first year students. CONCLUSIONS: Even though physicians and medical students agree that 'Responsibility' is the most important competency for entrustment decisions in the first year of residency, medical students rate competencies regarding patient communication very highly while physicians rate competencies required for patient managements significantly higher for entrustment decision. Undergraduate medical curricula seem to prepare students well with respect to patient-centeredness but need to be developed more specifically to prepare students equally well for patient management competencies which are required in the first year of residency for entrustment decisions from the attendings perspective.

Competencies for first year residents - physicians' views from medical schools with different undergraduate curricula.

BACKGROUND: Frameworks like the CanMEDS model depicting professional roles and specific professional activities provide guidelines for postgraduate education. When medical graduates start their residency, they should possess certain competencies related to communication, management and professionalism while other competencies will be refined during postgraduate training. Our study aimed to evaluate the relevance of different competencies for a first year resident required for entrustment decision from the perspective of physicians from medical faculties with different undergraduate medical curricula. METHODS: Nine hundred fifty-two surgeons and internists from three medical schools with different undergraduate medical curricula were invited to rank 25 competencies according to their relevance for first year residents. The rankings were compared between universities, specialties, physicians' positions, and gender. RESULTS: Two hundred two physicians participated, 76 from Hamburg University, 44 from Oldenburg University, and 82 from Technical University Munich. No significant differences were found regarding the top 10 competencies relevant for first year residents between the universities. 'Responsibility' was the competency with the highest rank overall. Internists ranked 'Structure, work planning and priorities' higher while surgeons ranked 'Verbal communication with colleagues and supervisors' higher. Consultants evaluated 'Active listening to patients' more important than department directors and residents. Female physicians ranked 'Verbal communication with colleagues and supervisors' and 'Structure, work planning and priorities' significantly higher while male physicians ranked 'Scientifically and empirically grounded method of working' significantly higher. CONCLUSIONS: Physicians from universities with different undergraduate curricula principally agreed on the competencies relevant for first year residents. Some differences between physicians from different positions, specialties, and gender were found. These differences should be taken into account when planning competence-based postgraduate education training programs.