A competency based selection procedure for Dutch postgraduate GP training: a pilot study on validity and reliability.

UNLABELLED: Abstract Background: Historically, semi-structured interviews (SSI) have been the core of the Dutch selection for postgraduate general practice (GP) training. This paper describes a pilot study on a newly designed competency-based selection procedure that assesses whether candidates have the competencies that are required to complete GP training. OBJECTIVES: The objective was to explore reliability and validity aspects of the instruments developed. METHODS: The new selection procedure comprising the National GP Knowledge Test (LHK), a situational judgement tests (SJT), a patterned behaviour descriptive interview (PBDI) and a simulated encounter (SIM) was piloted alongside the current procedure. Forty-seven candidates volunteered in both procedures. Admission decision was based on the results of the current procedure. RESULTS: Study participants did hardly differ from the other candidates. The mean scores of the candidates on the LHK and SJT were 21.9 % (SD 8.7) and 83.8% (SD 3.1), respectively. The mean self-reported competency scores (PBDI) were higher than the observed competencies (SIM): 3.7(SD 0.5) and 2.9(SD 0.6), respectively. Content-related competencies showed low correlations with one another when measured with different instruments, whereas more diverse competencies measured by a single instrument showed strong to moderate correlations. Moreover, a moderate correlation between LHK and SJT was found. The internal consistencies (intraclass correlation, ICC) of LHK and SJT were poor while the ICC of PBDI and SIM showed acceptable levels of reliability. CONCLUSION: Findings on content validity and reliability of these new instruments are promising to realize a competency based procedure. Further development of the instruments and research on predictive validity should be pursued.

Assessment of CanMEDS roles in postgraduate training: the validation of the Compass.

OBJECTIVE: In medical education the focus has shifted from gaining knowledge to developing competencies. To effectively monitor performance in practice throughout the entire training, a new approach of assessment is needed. This study aimed to evaluate an instrument that monitors the development of competencies during postgraduate training in the setting of training of general practice: the Competency Assessment List (Compass). METHODS: The distribution of scores, reliability, validity, responsiveness and feasibility of the Compass were evaluated. RESULTS: Scores of the Compass ranged from 1 to 9 on a 10-point scale, showing excellent internal consistency ranging from .89 to .94. Most trainees showed improving ratings during training. Medium to large effect sizes (.31-1.41) were demonstrated when we compared mean scores of three consecutive periods. Content validity of the Compass was supported by the results of a qualitative study using the RAND modified Delphi Method. The feasibility of the Compass was demonstrated. CONCLUSION: The Compass is a competency based instrument that shows in a reliable and valid way trainees' progress towards the standard of performance. PRACTICE IMPLICATIONS: The programmatic approach of the Compass could be applied in other specialties provided that the instrument is tailored to specific needs of that specialism.

Selection for Dutch postgraduate GP training; time for improvement.

BACKGROUND: In the Netherlands we select candidates for the postgraduate GP training by assessing personal qualities in interviews. Because of differences in the ratio of number of candidates and number of vacancies between the eight departments of GP training we questioned whether the risk of being rejected diverged amongst them. OBJECTIVE: The research question of this study was to which degree department of choice, candidates' characteristics and qualities assessed during interviews explain admission into GP training. METHODS: A nationwide observational study was conducted of all candidates who applied for postgraduate GP training in 2009/ 2010. Application ratio per department, candidates' characteristics (gender, age, region of medical school and times of application) and qualities (motivation, orientation on the job, personal attributes and learning needs) were collected. Outcome measures were admission to interview and admission to GP training. RESULTS: The study population addressed 542 candidates. Sixty three candidates were rejected on application letter (11.6%). So 479 candidates were admitted to the interview, of which 340 were admitted to the GP training (71%). Gender and region of medical school outside North West Europe were associated with admission to the interview. Department of choice had a strong association with admission in both stages (RR: 0.30 to 0.74; 0.20 to 0.79 respectively), while candidates' qualities explained admission (RR: 1.09- 1.25) as well. CONCLUSION: The influence of department of choice yields doubts about fairness of the procedure. So advantages and disadvantages of a national procedure are discussed as well as those of a competency based procedure.

Changes in the quality of doctor-patient communication between 1982 and 2001: an observational study on hypertension care as perceived by patients and general practitioners.

Background The rise of evidence-based medicine may have implications for the doctor-patient interaction. In recent decades, a shift towards a more task-oriented approach in general practice indicates a development towards more standardised healthcare. Objective To examine whether this shift is accompanied by changes in perceived quality of doctor-patient communication. Design GP observers and patient observers performed quality assessments of Dutch General Practice consultations on hypertension videotaped in 1982-1984 and 2000-2001. In the first cohort (1982-1984) 81 patients were recorded by 23 GPs and in the second cohort (2000-2001) 108 patients were recorded by 108 GPs. The GP observers and patient observers rated the consultations on a scale from 1 to 10 on three quality dimensions: medical technical quality, psychosocial quality and quality of interpersonal behaviour. Multilevel regression analyses were used to test whether a change occurred over time. Results The findings showed a significant improvement over time on all three dimensions. There was no difference between the quality assessments of GP observers and patient observers. The three different dimensions were moderately to highly correlated and the assessments of GP observers showed less variability in the second cohort. Conclusions Hypertension consultations in general practice in the Netherlands received higher quality assessments by general practitioners and patients on medical technical quality, psychosocial quality and the quality of interpersonal behaviour in 2000-2001 as compared with the 1980s. The shift towards a more task-oriented approach in hypertension consultations does not seem to detract from individual attention for the patient. In addition, there is less variation between general practitioners in the quality assessments of more recent consultations. The next step in this line of research is to unravel the factors that determine patients' quality assessments of doctor-patient communication.

Teachers' ideas versus experts' descriptions of 'the good teacher' in postgraduate medical education: implications for implementation. A qualitative study.

BACKGROUND: When innovations are introduced in medical education, teachers often have to adapt to a new concept of what being a good teacher includes. These new concepts do not necessarily match medical teachers' own, often strong beliefs about what it means to be a good teacher.Recently, a new competency-based description of the good teacher was developed and introduced in all the Departments of Postgraduate Medical Education for Family Physicians in the Netherlands. We compared the views reflected in the new description with the views of teachers who were required to adopt the new framework. METHODS: Qualitative study. We interviewed teachers in two Departments of Postgraduate Medical Education for Family Physicians in the Netherlands. The transcripts of the interviews were analysed independently by two researchers, who coded and categorised relevant fragments until consensus was reached on six themes. We investigated to what extent these themes matched the new description. RESULTS: Comparing the teachers' views with the concepts described in the new competency-based framework is like looking into two mirrors that reflect clearly dissimilar images. At least two of the themes we found are important in relation to the implementation of new educational methods: the teachers' identification and organisational culture. The latter plays an important role in the development of teachers' ideas about good teaching. CONCLUSIONS: The main finding of this study is the key role played by the teachers' feelings regarding their professional identity and by the local teaching culture in shaping teachers' views and expectations regarding their work. This suggests that in implementing a new teaching framework and in faculty development programmes, careful attention should be paid to teachers' existing identification model and the culture that fostered it.

Behavioural elements of professionalism: assessment of a fundamental concept in medical care.

BACKGROUND: The Nijmegen Professionalism Scale, an instrument for assessing professional behaviour of general practitioner (GP) trainees, consists of four domains: professional behaviour towards patients, other professionals, society and oneself. The purpose of the instrument is to provide formative feedback. AIM: The aim of this study was to examine the psychometric properties of the Nijmegen Professionalism Scale. METHODS: Both GP trainers and their GP trainees participated. Factor analysis was conducted for each domain. Factor structures of trainee and trainer groups were compared. Measure of congruence used was Tucker's phi. Cronbach's alpha was used to establish reliability. RESULTS: Factor structures of the instrument used by GP trainers and trainees were similar. Two factors for each domain were found: domain 1, Respecting patient's interests and Professional distance; domain 2, Collaboration skills and Management skills; domain 3, Responsibility and Quality management; and domain 4, Reflection and learning and Dealing with emotions. Congruence measures were substantial (>0.90). Reliability ranged from 0.78 to 0.95. CONCLUSION: This study to validate the instrument represents one further step. To construct a sound validity argument, a much broader range of evidence is required. Nevertheless, this study shows that the Nijmegen Professionalism Scale is a reliable tool for assessing professional behaviour.

The impact of assessing simulated bad news consultations on medical students' stress response and communication performance.

Seventy second-year medical students volunteered to participate in a study with the aim of evaluating the impact of the assessment of simulated bad news consultations on their physiological and psychological stress and communication performance. Measurements were taken of salivary cortisol, systolic and diastolic blood pressure, heart rate, state anxiety and global stress using a Visual Analogue Scale. The subjects were asked to take three salivary cortisol samples on the assessment day as well as on a quiet control day, and to take all other measures 5 min before and 10 min after conducting the bad news consultation. Consultations were videotaped and analyzed using the information-giving subscale of the Amsterdam Attitude and Communication Scale (AACS), the Roter Interaction Analysis System (RIAS), and the additional non-verbal measures, smiling, nodding and patient-directed gaze. MANOVA repeated measurements were used to test the difference between the cortisol measurements taken on the assessment and the control day. Linear regression analysis was used to determine the association between physiological and psychological stress measures and the students' communication performance. The analyses were restricted to the sample of 57 students who had complete data records. In anticipation of the communication assessment, cortisol levels remained elevated, indicating a heightened anticipatory stress response. After the assessment, the students' systolic blood pressure, heart rate, globally assessed stress level and state anxiety diminished. Pre-consultation stress did not appear to be related to the quality of the students' communication performance. Non-verbal communication could be predicted by pre-consultation physiological stress levels in the sense that patient-directed gaze occurred more often the higher the students' systolic blood pressure and heart rate. Post-consultation heart rate remained higher the more often the students had looked at the patient and the more information they had provided. However, the heart rate appeared to diminish the more often the students had reassured the patient. These findings suggest that in evaluating students' communication performance there is a need to take their stress levels into account.

Cultural differences in managing information during medical interaction: how does the physician get a clue?

OBJECTIVE: Consultations of ethnic-minority patients tend to result in poor mutual understanding between doctor and patient, which may have serious consequences for health care. For good communication, physicians have strong devices at their disposal to manage the information, such as agenda-setting and structuring the interview into segments. What are the cultural differences in the managing of information in medical conversation? What is the relation with level of mutual understanding? METHODS: Data of 103 transcripts of video-registered medical interviews (56 non-Western and 47 Dutch patients) were sequentially analysed, focusing on relevant segments of the medical interview (medical history, diagnosis and conclusion) and on agenda-setting. RESULTS: Physicians set the agenda and lead the conversation firmly forward, while a considerable number of patients (mainly Dutch) 'put on the brakes'. The majority of the medical conversations was traditional (37%) or cooperative (37%), while another 25% was more or less conflicting or complaintive in nature. Interviews of ethnic-minority patients were mostly traditional or cooperative, while Dutch patients showed a variety of types, especially in cases of poor mutual understanding. Further, conversational symmetry between patient and physician has increased over the years, due to the importance attached to patient autonomy. CONCLUSION: Physicians receive different conversational clues from Dutch and ethnic-minority patients in case of poor mutual understanding. PRACTICE IMPLICATIONS: This points to the necessity for physicians as well as patients to become culturally competent.

Cultural diversity in patient participation: the influence of patients' characteristics and doctors' communicative behaviour.

OBJECTIVE: The primary goal of this study was to examine the extent to which patient participation during medical visits is influenced by patients' ethnic background, patients' culture-related characteristics (e.g. acculturation, locus of control, cultural views) and features of doctors' communicative behaviour. Furthermore, the mutual influence between patients' participatory behaviour and doctors' communicative behaviour was investigated. An additional goal was to identify the independent contribution of these variables to the degree of patient satisfaction and mutual understanding between GP and patient. METHODS: Communicative behaviour of patients (n=103) and GPs (n=29) was analysed with Roter's Interaction Analysis System, frequency of patient questions and patients' assertive utterances (e.g. making requests, suggesting alternative treatment options). Additional data were gathered using GP and patient questionnaires after the consultations. RESULTS: Results show that non-Western ethnic minority patients display less participatory behaviour during medical consultations than Dutch patients. GPs' affective verbal behaviour had most effect on degree of patient participation and patient satisfaction. Regression analyses indicate a significant mutual influence between patients' verbal behaviour and GPs' verbal behaviour. CONCLUSION: Overall, results of this study show some important differences between Dutch and non-Western ethnic minority patients in degree of patient participation. Furthermore, our results indicate that patient participation encompasses several aspects that are not necessarily interrelated. PRACTICE IMPLICATIONS: The necessity for continued education of GPs' communicative skills, particularly when dealing with non-Western ethnic minority patients, is reflected in the strong influence of GP's affective verbal behaviour on both patient participation and their satisfaction with the medical encounter.

Shifts in doctor-patient communication between 1986 and 2002: a study of videotaped general practice consultations with hypertension patients.

BACKGROUND: Departing from the hypotheses that over the past decades patients have become more active participants and physicians have become more task-oriented, this study tries to identify shifts in GP and patient communication patterns between 1986 and 2002. METHODS: A repeated cross-sectional observation study was carried out in 1986 and 2002, using the same methodology. From two existing datasets of videotaped routine General Practice consultations, a selection was made of consultations with hypertension patients (102 in 1986; 108 in 2002). GP and patient communication was coded with RIAS (Roter Interaction Analysis System). The data were analysed, using multilevel techniques. RESULTS: No gender or age differences were found between the patient groups in either study period. Contrary to expectations, patients were less active in recent consultations, talking less, asking fewer questions and showing less concerns or worries. GPs provided more medical information, but expressed also less often their concern about the patients' medical conditions. In addition, they were less involved in process-oriented behaviour and partnership building. Overall, these results suggest that consultations in 2002 were more task-oriented and businesslike than sixteen years earlier. CONCLUSION: The existence of a more equal relationship in General Practice, with patients as active and critical consumers, is not reflected in this sample of hypertension patients. The most important shift that could be observed over the years was a shift towards a more businesslike, task-oriented GP communication pattern, reflecting the recent emphasis on evidence-based medicine and protocolized care. The entrance of the computer in the consultation room could play a role. Some concerns may be raised about the effectiveness of modern medicine in helping patients to voice their worries.

Interdisciplinary preoperative patient education in cardiac surgery.

BACKGROUND: In 1998, we carried out a study of interdisciplinary preoperative education in cardiac surgery given by nurses, physicians and health educators. Overlaps were found in gathering and providing information by physicians, nurses and health educators, and gaps were found in providing psychosocial information and emotional support. Based on these findings, an information protocol was developed. AIM: This paper reports a study examining the effects of the implementation of the information protocol on the content and process of preoperative education. METHODS: Dialogues between health educators and patients were videotaped at the preoperative clinic (n = 54) and on the day of admission (n = 53), and analysed using a checklist of 123 specific topics. RESULTS: The information given by health educators at the outpatient clinic and on the day of admission accorded with the information protocol to a large extent. There was also an increase in the number of psychosocial issues raised. Nurses raised significantly more psychosocial issues in comparison with before implementation of the protocol. After implementation, patients spent approximately 3 minutes less talking with the health educator and about 7 minutes less talking with a nurse. This suggests that on the day of admission a more time-efficient co-ordination in patient education was achieved. CONCLUSIONS: Implementation of the information protocol led to a better interdisciplinary division of labour. The education is tailored more to the needs of the patient, and psychosocial items are mentioned more frequently. This straightforward intervention gave very positive results. Inconsistencies, gaps and overlaps in information provision can be avoided by the unambiguous delineation of responsibilities and tasks in information provision by different health care providers.