Evaluating real-patient learning in medical education - Hungarian validation of the Manchester Clinical Placement Index.

Introduction: The Manchester Clinical Placement Index (MCPI) is an instrument to measure medical undergraduates' real-patient learning in communities of practice both in hospital and in GP placements. Its suitability to evaluate the quality of placement learning environments has been validated in an English-language context; however, there is a lack of evidence for its applicability in other languages. Our aim was to thoroughly explore the factor structure and the key psychometric properties of the Hungarian language version. Methods: MCPI is an 8-item, mixed-method instrument which evaluates the quality of clinical placements as represented by the leadership, reception, supportiveness, facilities and organization of the placement (learning environment) as well as instruction, observation and feedback (training) on 7-point Likert scales with options for free-text comments on the strengths and weaknesses of the given placement on any of the items. We collected data online from medical students in their preclinical (1st, 2nd) as well as clinical years (4th, 5th) in a cross-sectional design in the academic years 2019-2020 and 2021-2022, by the end of their clinical placements. Our sample comprises data from 748 medical students. Exploratory and confirmatory factor analyses were performed, and higher-order factors were tested. Results: Although a bifactor model gave the best model fit (RMSEA = 0.024, CFI = 0.999, and TLI = 0.998), a high explained common variance (ECV = 0.82) and reliability coefficients (ωH = 0.87) for the general factor suggested that the Hungarian version of the MCPI could be considered unidimensional. Individual application of either of the subscales was not supported statistically due to their low reliabilities. Discussion: The Hungarian language version of MCPI proved to be a valid unidimensional instrument to measure the quality of undergraduate medical placements. The previously reported subscales were not robust enough, in the Hungarian context, to distinguish, statistically, the quality of learning environments from the training provided within those environments. This does not, however, preclude formative use of the subscales for quality improvement purposes.

Desempenho acadêmico de doutores egressos do Programa de Pós-Graduação em Medicina: Ciências Médicas da Faculdade de Medicina da UFRGS / Academic performance of doctoral graduates from the Graduate Program in Medicine: Medical Sciences at UFRGS School of Medicine

Introdução: embora, no Brasil, muito tenha sido feito para estimular a criação e manutenção de programas de pós-graduação ainda há carência de meios para avaliar e para mensurar a evolução profissional dos egressos. O objetivo é compreender a trajetória dos Doutores formados pelo Programa de Pós-Graduação em Medicina Ciências Médicas (PPGCM), Faculdade de Medicina, Universidade Federal do Rio Grande do Sul (UFRGS) e propor um índice de desempenho acadêmico e de inserção profissional. Métodos: foi realizado um estudo transversal. Foram coletados dados dos Currículos Lattes dos egressos de doutorado do Programa de Pós-Graduação em Medicina: Ciências Médicas (PPGCM), no período de 1987 a 2014. Foram extraídos apenas os dados pertinentes a sua origem, curso de graduação e de mestrado, produção intelectual, inserção acadêmica e a posição profissional. Resultados: foram analisados 324 doutores egressos, 221 eram graduados em Medicina. Foram utilizadas as variáveis: graduação em medicina, vínculo laboral em universidade pública, exerce atividade como orientador em nível de doutorado, exerce atividade como orientador em nível de mestrado, bolsista de produtividade em pesquisa do CNPq e possui registro do índice H no currículo Lattes. O coeficiente de determinação (R2) demonstrou que as variáveis incluídas no modelo explicam 99% do modelo hierárquico. Conclusão: os resultados demonstram que os egressos têm contribuído para a construção de conhecimento qualificado disseminado em jornais de circulação internacional, formação de recursos humanos, interagir e criar parcerias em seu ambiente de trabalho, gerenciar e repassar o conhecimento, bem como para fomentar o conhecimento na área das ciências médicas com relevância regional, nacional e internacional. (AU)

Introduction: Although, in Brazil, much has been done to encourage the creation and maintenance of graduate programs, there remains a lack of means to assess and measure the professional outcomes of graduates. The objective is to understand the trajectory of doctoral graduates from the Graduate Program in Medicine: Medical Sciences (PPGCM), School of Medicine, Universidade Federal do Rio Grande do Sul (UFRGS), and propose an index for academic performance and professional placement. Methods: A cross-sectional study was conducted. Data available at CV Lattes database of the National Council for Scientific and Technological Development (CNPq) were collected on doctoral graduates from the PPGCM, from 1987 to 2014. Only data pertaining to their origin, undergraduate and master's degrees, intellectual output, academic performance, and professional placement were retrieved. Results: Data on 324 doctoral graduates were analyzed, 221 had graduated in medicine. Undergraduate degree, employment at a public university, working as a doctoral-level advisor, working as a master's-level advisor, CNPq research productivity grantee, and having a recorded h-index in the Lattes curriculum were collected. The coefficient of determination (R 2 ) demonstrated that the variables included in the model explain 99% of the hierarchical model. Conclusion: The results show that the graduates have contributed to developing qualified knowledge disseminated in international newspapers, training human resources, interacting and creating partnerships in the workplace, managing and passing on knowledge, as well as fostering knowledge in the area of medical sciences with regional, national, and international relevance. (AU)

Medical school clinical placements - the optimal method for assessing the clinical educational environment from a graduate entry perspective.

BACKGROUND: Educational environment is a strong determinant of student satisfaction and achievement. The learning environments of medical students on clinical placements are busy workplaces, composed of many variables. There is no universally accepted method of evaluating the clinical learning environment, nor is there consensus on what concepts or aspects should be measured. The aims of this study were to compare the Dundee ready educational environment measure (DREEM - the current de facto standard) and the more recently developed Manchester clinical placement index (MCPI) for the assessment of the clinical learning environment in a graduate entry medical student cohort by correlating the scores of each and analysing free text comments. This study also explored student perceptionof how the clinical educational environment is assessed. METHODS: An online, anonymous survey comprising of both the DREEM and MCPI instruments was delivered to students on clinical placement in a graduate entry medical school. Additional questions explored students' perceptions of instruments for giving feedback. Numeric variables (DREEM score, MCPI score, ratings) were tested for normality and summarised. Pearson's correlation coefficient was used to measure the strength of the association between total DREEM score and total MCPI scores. Thematic analysis was used to analyse the free text comments. RESULTS: The overall response rate to the questionnaire was 67% (n = 180), with a completed response rate for the MCPI of 60% (n = 161) and for the DREEM of 58% (n = 154). There was a strong, positive correlation between total DREEM and MCPI scores (r = 0.71, p < 0.001). On a scale of 0 to 7, the mean rating for how worthwhile students found completing the DREEM was 3.27 (SD 1.41) and for the MCPI was 3.49 (SD 1.57). 'Finding balance' and 'learning at work' were among the themes to emerge from analysis of free text comments. CONCLUSIONS: The present study confirms that DREEM and MCPI total scores are strongly correlated. Graduate entry students tended to favour this method of evaluation over the DREEM with the MCPI prompting rich description of the clinical learning environment. Further study is warranted to determine if this finding is transferable to all clinical medical student cohorts.

Can less be more? Comparison of an 8-item placement quality measure with the 50-item Dundee Ready Educational Environment Measure (DREEM).

Clinical clerks learn more than they are taught and not all they learn can be measured. As a result, curriculum leaders evaluate clinical educational environments. The quantitative Dundee Ready Environment Measure (DREEM) is a de facto standard for that purpose. Its 50 items and 5 subscales were developed by consensus. Reasoning that an instrument would perform best if it were underpinned by a clearly conceptualized link between environment and learning as well as psychometric evidence, we developed the mixed methods Manchester Clinical Placement Index (MCPI), eliminated redundant items, and published validity evidence for its 8 item and 2 subscale structure. Here, we set out to compare MCPI with DREEM. 104 students on full-time clinical placements completed both measures three times during a single academic year. There was good agreement and at least as good discrimination between placements with the smaller MCPI. Total MCPI scores and the mean score of its 5-item learning environment subscale allowed ten raters to distinguish between the quality of educational environments. Twenty raters were needed for the 3-item MCPI training subscale and the DREEM scale and its subscales. MCPI compares favourably with DREEM in that one-sixth the number of items perform at least as well psychometrically, it provides formative free text data, and it is founded on the widely shared assumption that communities of practice make good learning environments.