ABSTRACT
La evaluación del desempeño del docente clínico permite proporcionar feedback para motivarlo a mejorar sus habilidades docentes. Se realizó la validación transcultural del Cuestionario de Enseñanza Clínica de Maastricht, mediante la guía internacional para la adaptación de los cuestionarios. Se investigó validez de contenido, proceso de respuesta, constructo y confiabilidad. Luego de la adaptación, residentes de dos hospitales universitarios evaluaron a 187 docentes clínicos. La validación del contenido y del proceso de respuesta fue adecuada. En el análisis factorial confirmatorio, todos los índices y criterios se ajustaron bien con el modelo de 5 factores y 14 ítems. El alfa de Cronbach fue de 0,80. El coeficiente G fue > 0,70, con bajos valores de varianza del error absoluto. Fueron necesarias 6 evaluaciones por docente para una evaluación confiable en cada dominio y en el desempeño global. La versión en castellano del Cuestionario es válida y confiable para evaluar a los docentes.
The evaluation of the clinical teacher's performance provides feedback to motivate them to improve their teaching skills. To perform the cross-cultural adaptation of the Maastricht Clinical Teaching Questionnaire, the International Guide for the Adaptation of the Questionnaires was followed. The validity of content, response process, construct and reliability were investigated. After cross-cultural adaption, residents of two University hospitals evaluated 187 clinical teachers. Content and answering process were validated. In the confirmatory factor analysis, all indexes and criteria for a good fit suited the 5 factors and 14 items model. The Cronbach's alpha coefficient was 0.80. The G coefficient was > 0.70, with low variance of the absolute error. Every clinical teacher should receive at least 6 evaluations to achieve a reliable evaluation of every domain and of their global performance. The Spanish version of the questionnaire is a valid, reliable instrument for medical residents to evaluate teachers.
Subject(s)
Humans , Cross-Cultural Comparison , Surveys and Questionnaires , Reproducibility of Results , Adaptation to Disasters , Education, MedicalABSTRACT
The evaluation of the clinical teacher's performance provides feedback to motivate them to improve their teaching skills. To perform the cross-cultural adaptation of the Maastricht Clinical Teaching Questionnaire, the International Guide for the Adaptation of the Questionnaires was followed. The validity of content, response process, construct and reliability were investigated. After cross-cultural adaption, residents of two University hospitals evaluated 187 clinical teachers. Content and answering process were validated. In the confirmatory factor analysis, all indexes and criteria for a good fit suited the 5 factors and 14 items model. The Cronbach's alpha coefficient was 0.80. The G coefficient was > 0.70, with low variance of the absolute error. Every clinical teacher should receive at least 6 evaluations to achieve a reliable evaluation of every domain and of their global performance. The Spanish version of the questionnaire is a valid, reliable instrument for medical residents to evaluate teachers.
La evaluación del desempeño del docente clínico permite proporcionar feedback para motivarlo a mejorar sus habilidades docentes. Se realizó la validación transcultural del Cuestionario de Enseñanza Clínica de Maastricht, mediante la guía internacional para la adaptación de los cuestionarios. Se investigó validez de contenido, proceso de respuesta, constructo y confiabilidad. Luego de la adaptación, residentes de dos hospitales universitarios evaluaron a 187 docentes clínicos. La validación del contenido y del proceso de respuesta fue adecuada. En el análisis factorial confirmatorio, todos los índices y criterios se ajustaron bien con el modelo de 5 factores y 14 ítems. El alfa de Cronbach fue de 0,80. El coeficiente G fue > 0,70, con bajos valores de varianza del error absoluto. Fueron necesarias 6 evaluaciones por docente para una evaluación confiable en cada dominio y en el desempeño global. La versión en castellano del Cuestionario es válida y confiable para evaluar a los docentes.
Subject(s)
Cultural Characteristics , Education, Medical, Graduate/standards , Faculty, Medical , Professional Competence , Surveys and Questionnaires , Humans , Reproducibility of ResultsABSTRACT
INTRODUCTION: Judgment in the face of uncertainty is an important dimension of expertise and clinical competence. However, it is challenging to conceive continuing professional development (CPD) initiatives aimed at helping physicians enhance their clinical judgment skills in ill-defined situations. We present an online script concordance-based CPD program (the Practicum Script Concordance Test, copyright ©2006 by Practicum Foundation), a tool that can be used to support health professionals in the development of their reflective clinical reasoning ability. We describe the rationale and principles and report on the implementation of 2 online programs based on this new CPD initiative. METHOD: The Practicum Script Concordance Test program consists of daily testing and feedback over the course of a year using SCT items. Feedback is both global (eg, health professionals are told their cumulative mean score) and specific (eg, they can view the expert panel's responses together with their justifications for their answers). Participants have the option of contacting a personal tutor, to whom they can send questions. Data regarding feasibility, participation, and acceptability were collected. RESULTS: Initial implementation took place in Mexico where 1901 physicians (1349 paediatricians, 552 cardiologists) were enrolled in Practicum programs. Around 70% of those enrolled pursued the program and were very satisfied with its format and content. The online format was an important factor in the development and maintenance of the programs. Dropouts had issues with the SCT concept and the time required to participate. DISCUSSION: The on-line Practicum Script Concordance Test program was designed to foster expertise development based on practice, reflection and feedback. Although further research is needed to examine its impact physicians' practice and ultimately on patient outcomes, it is an original and promising development in CPD.
Subject(s)
Education, Medical, Continuing/methods , Educational Measurement/methods , Internet , Adult , Clinical Competence , Female , Humans , Male , Mexico , Middle Aged , Program Development , Self EfficacyABSTRACT
Background: Bridging the theory of psychometric assessment to an evaluation instrument is a challenging and continuous task. Such a task was completed for the SP-based OSCE licensing examination used in Quebec since 1990. Purpose: To review the comprehensive series of psychometric studies undertaken on the examination based on the theory of psychometric assessment. Methods: The examination is a 5.5 hour-long OSCE composed of 26 clinical cases of 7, 10 or 15 minutes' duration. Candidates' total scores and case scores are used for licensing decisions. Reliability coefficients were obtained as well as generalizability studies to assess the impact of examination sites (4), tracks (14) and languages (French and English) on candidates' results. Its content validity was assessed by experts as well as by practicing family physicians, examiners and exam coordinators. Concurrent validity was assessed. Through differential item functioning (DIF) statistical approach, possible gender biases were analyzed. The predictive validity of the examination scores on selected aspects of practice was analyzed as well as the impact of the examination on the teaching and learning process. Results: These studies confirmed the validity, reliability and generalizability of the examination scores. No biases were demonstrated. Examination scores predicted elements of practice. Educational impact was present. Conclusion: This study confirmed the quality of the examination, ensured its continuous improvement and supported the inferences made from the examination results.
ABSTRACT
The 'Simulated Surgery' is an alternative consulting skills component of the Membership examination of the Royal College of General Practitioners, which is the professional certifying examination for GP registrars (family medicine residents) in the UK. It consists of a 20 station OSCE and is taken by a small cohort of candidates (10--30) who are unable to provide a videotape of their patient interviews for assessment. The passmark for this examination has been set by a modified contrasting groups method, in which all the examiners make pass/fail judgements on all the candidates' performance by reviewing their whole-test grades. A consistent passmark was obtained for two different cohorts and this method should allow a constant passing standard to be maintained under changing circumstances in the future.
