Facilitators and barriers to a nationwide implementation of competency-based postgraduate medical curricula: a qualitative study.

BACKGROUND: Postgraduate medical education (PGME) curricula are being redesigned across the western world. AIM: This study examined the implementation process (what works where and why) of new competency-based PGME curricula and relevant factors influencing this process. METHODS: In a nationwide project (2006-2010) in the Netherlands, competency-based PGME curricula were implemented for residents in Pediatrics and Obstetrics & Gynecology. The authors conducted 25 semi-structured interviews and used a multi-level theoretical framework to guide coding. RESULTS: The implementation process proved to be highly dynamic, non-linear, and influenced by many factors. These could be divided into attributes of the innovations/adopters, the implementation process, and the organization. The context determined the speed, quality, and direction of the process and how a factor affected the process. CONCLUSIONS: We identified specific features of PGME innovation: the challenge of implementing other competencies than that of the medical expert; the importance of regional implementation strategies and educational support; the balance between training and patient care; and the need for regional inter-organizational networks of hospitals. The authors recommend: design the curriculum with the needs of the users in mind; facilitate knowledge sharing; organize educational support; translate the national curriculum to the local workplace; and promote regional inter-organizational networks between hospitals.

Attending doctors' perspectives on how residents learn.

CONTEXT: Graduate medical education is currently facing major educational reforms. There is a lack of empirical evidence in the literature about the learning processes of residents in the clinical workplace. This qualitative study uses a 'grounded theory' approach to continue the development of a theoretical framework of learning in the clinical workplace by adding the perspective of attending doctors. METHODS: A total of 21 Dutch attending doctors involved in the training of residents in obstetrics and gynaecology participated in 1 of 3 focus group sessions. They discussed their perceptions of how residents learn and what factors influence residents' learning. A grounded theory approach was used to analyse the transcribed discussions. RESULTS: Three related themes emerged. The first concerned the central role of participation in work-related activities: according to attending doctors, residents learn by tackling the everyday challenges of clinical work. The second involved the ways in which attending doctors influence what residents learn from work-related activities. The final theme focused on attending doctors' views of the essential characteristics of residents and their development during residency. CONCLUSIONS: Attending doctors' perspectives complement current insights derived from similar research among residents and from related literature. As part of an ongoing effort to further develop understanding of how residents learn, this study adds several ways in which attending doctors strive to combine guidance in both patient care and resident training. Furthermore, attending doctors' perspectives draw attention to other aspects of learning in the clinical workplace, such as the role of confidence and the balance between supervision and independence.

How residents learn: qualitative evidence for the pivotal role of clinical activities.

OBJECTIVES: Medical councils worldwide have outlined new standards for postgraduate medical education. This means that residency programmes will have to integrate modern educational views into the clinical workplace. Postgraduate medical education is often characterised as a process of learning from experience. However, empirical evidence regarding the learning processes of residents in the clinical workplace is lacking. This qualitative study sought insight into the intricate process of how residents learn in the clinical workplace. METHODS: We carried out a qualitative study using focus groups. A grounded theory approach was used to analyse the transcribed tape recordings. A total of 51 obstetrics and gynaecology residents from teaching hospitals and affiliated general hospitals participated in 7 focus group discussions. Participants discussed how they learn and what factors influence their learning. RESULTS: An underlying theoretical framework emerged from the data, which clarified what happens when residents learn by doing in the clinical workplace. This framework shows that work-related activities are the starting point for learning. The subsequent processes of 'interpretation' and 'construction of meaning' lead to refinement and expansion of residents' knowledge and skills. Interaction plays an important role in the learning process. This is in line with both cognitivist and sociocultural views on learning. CONCLUSIONS: The presented theoretical framework of residents' learning provides much needed empirical evidence for the actual learning processes of residents in the clinical workplace. The insights it offers can be used to exploit the full educational potential of the clinical workplace.

Assessment in problem-based learning (PBL).

Problem-based learning (PBL), a relatively new approach to teaching, has been introduced in several of the medical schools in the Southeast Pacific region. This approach emphasises active, self-directed learning and puts the emphasis on the individual student and his/her role in the educational process. It is well known that assessment plays a large role in influencing student learning behaviour. Therefore, it is important that the assessment process not hamper learning or adversely affect attainment of the goals of the curriculum. If student behaviours are directed toward achieving success on the assessments, instructors' efforts to create a climate of self-directed learning and individual responsibility will be frustrated. This article provides information on the different classes of assessment vehicles available to assess the various domains of medical competence. We pay special attention to the critical relationship between the educational goals of problem-based learning and the different assessment instruments used. We briefly describe some assessment instruments and how it can relate to these goals in a positive manner. We conclude that no assessment instrument is ideal and that each has unique inherent deficiencies. As a consequence, a successful student assessment process requires use of several complementary instruments.

Sequential testing in the assessment of clinical skills.

PURPOSE: Educators who use standardized-patient-based (SP-based) tests may save resources by using sequential testing. In this approach, students take a short screening test; only those who fail take a second test. This study investigated whether sequential testing increases efficiency with only a minor decrease of validity. METHOD: In 1994-95, first- through fourth-year (Group 1) and sixth-year (Group 2) medical students at the University of Maastricht took SP-based tests. Each test took two days. In a simulation experiment based on the data from those tests, the authors considered the first day as the screening test and the second day as the second test. They investigated efficiency and validity as a function of the cutoff score of the screening test. They developed and evaluated a new method to determine the optimum cutoff score of the screening test, a method based on minimization of the loss represented by the (weighted) numbers of false positives and negatives in the screening test. RESULTS: The negative predictive value (probability that a student would fail the complete test if he or she had failed the screening test) was low (<60%), while the positive predictive value was high (>96%). Accordingly, stringent pass/fail cutoff scores in the screening test (75% for Group 1 and 80% for Group 2) produced optimum results. Using those cutoff values, only 26% and 11% of 'the students would have had to take the complete test to get a "true" score, while only 0.2% and 0.0% of the students who passed the screening test went on to fail the complete test (false positives). CONCLUSIONS: In a sequential SP-based test, the pass/fail cutoff score of the screening test should be stringent. This can considerably reduce testing time (30% to 40%), while keeping the percentage of false positives at an acceptably low level of less than 0.2%. As an alternative to receiver operator characteristic analysis, minimization of the loss function was found to be an appropriate method to determine the optimum cutoff value of the screening test.

Assessment of competence in technical clinical skills of general practitioners.

Technical clinical procedures constitute an important part of the work of general practitioners. Assessment of competence in the relevant skills is important from the perspective of quality assurance. In this study, the psychometric characteristics of three different methods for assessment of competence in technical clinical skills in general practice were evaluated. A performance-based test (8 stations), a written knowledge test of skills (125 items) and a self-assessment questionnaire (41 items) on technical clinical skills were administered to 49 GPs and 47 trainees in general practice. The mean scores on the performance-based test and the written knowledge test of skills showed no substantial differences between GPs and trainees, whereas the GPs scored higher on the self-assessment questionnaire. While the correlation of the score on the knowledge test of skills with the score on the performance-based test was moderately high, the score on the self-assessment questionnaire showed a rather low correlation with the performance-based test. Although performance-based testing is obviously the best method to assess proficiency in hands-on skills, a written test can serve as a reasonable alternative, particularly for screening and research purposes.