Analysis of the effects of the teaching reform to strengthen the integration of basic medicine and clinical medicine through the National Medical Licensing Examination scores / 中华医学教育探索杂志

Objective:To investigate the effects of teaching reform to strengthen the integration of basic medicine and clinical medicine by analyzing the scores of the National Medical Licensing Examination from 2014 to 2018.Methods:Through the improvement of the training program, the establishment of subject integration courses, the standardization of case teaching mode and the establishment of comprehensive medical experiment courses, the teaching reform of the combination of basic medicine and clinical medicine was carried out. The differences in the scores of the National Medical Licensing Examination between the students from Dalian University and those from the whole country were compared, including the average total scores, the passing rates, the average mastery rates of the basic medicine, and the average mastery rates of the cognitive level of the basic medicine.Results:In 2014, 2015 and 2016, the average total scores of the students from Dalian University were 384.70±53.62, 395.45±50.49, and 401.43±50.88, respectively, which were lower than those from the whole country (396.15±58.85, 400.78±56.88, and 405.15±58.06). The average mastery rates of the basic medicine of the students from Dalian University were 56.28%, 62.45%, and 64.92%, respectively, lower than those from the whole country (58.41%, 62.46%, and 65.36%). The average mastery rates of the basic medicine of the students form Dalian University from 2014 to 2016 were lower than those from the whole country, and those were higher in 2017 to 2018 than the whole country. The average total scores of the students form Dalian University in 2017 and 2018 were 397.11 and 400.26 ± 61.15, respectively, higher than those from the whole country (388.91 and 397.01±58.35). The average mastery rates of the basic medicine of the students form Dalian University were 61.53% and 59.85%, respectively, higher than those from the whole country (58.64% and 58.38%). The total passing rates of the students from Dalian University in 2014 and 2016 were 91.11% and 72.73%, respectively, lower than those from the whole country (91.18% and 73.75%), while in 2015, 2017 and 2018 those were 76.47%, 77.78%, and 77.59%, respectively, all higher than those from the whole country (73.59%, 74.29%, and 76.89%). All the data showed a trend of increase with the teaching reform process.Conclusion:Strengthening the teaching reform of the combination of basic medicine and clinical medicine during the learning stage of basic medicine could improve students' learning interest and attention to the basic medical knowledge, strengthen the understanding of the relevance of basic medicine to clinical medicine, and play a positive role in the cultivation of medical students' post competency and the achievement of the goal of training applied medical talents.

Application and effects of objective structured clinical examination in the clinical ability assessment of stomatology undergraduates / 中华医学教育探索杂志

Objective:To explore the objective structured clinical examination (OSCE) on the improvement of clinical competence of stomatology undergraduates, and to promote their ability and skills of taking the National Medical Licensing Examination (NMLE).Methods:The OSCE was implemented to evaluate the clinical skills of stomatology undergraduates from Batch 2009 to Batch 2012. The OSCE results were analyzed and compared with the NMLE (Stomatology) results on the correlation coefficient analysis of annual passing rate.Results:The passing rate of practical skills in the NMLE (Stomatology) of Batch 2009 to Batch 2012 stomatology undergraduates was basically consistent with the excellent rate of OSCE, and the correlation coefficient was 0.94.Conclusion:It has not been proved that the OSCE assessment organized by our school can increase the passing rate of practical skills in the NMLE, but it has certainly helped students better understand and apply theoretical knowledge, and it plays a positive role in promoting students' future development.

Exames de licenciamento em Medicina - uma boa ideia para a formação médica no Brasil? / Medical licensing examinations - are they a good idea for medical education in Brazil? / Exámenes de licencia en Medicina - ¿Una buena idea para la formación médica en Brasil?

A sociedade brasileira vem discutindo, recentemente, a ideia de implementar um exame de licenciamento em Medicina (ELM) para médicos recém-formados no país. Para avaliar a adequação dessa proposta e sua real necessidade, analisa-se neste trabalho se os ELMs estabelecem eficazmente a aptidão dos recém-formados para exercer a Medicina e se os sistemas de avaliação vigentes no ensino médico brasileiro já o fazem. Usando o modelo baseado em competências como referencial, discute-se a experiência da comunidade internacional com os ELMs e as características das avaliações que se propõem a medir, no Brasil, a aptidão dos recém-formados. Conclui-se que, isoladamente, ambas as alternativas são ineficazes para cumprir esse objetivo. Propõe-se o desenvolvimento de um programa mais amplo de avaliação e enfatiza-se a importância da colaboração entre os atores envolvidos para aprimorar a qualidade das avaliações dos médicos recém-formados no Brasil.(AU)

The introduction a medical licensing examination (MLE) for the country's newly qualified doctors has been the subject of discussion recently in Brazil. To assess the suitability of and real need for this test, this article analyzes whether the MLE provides an effective measure of the aptitude of newly qualified doctors and whether the evaluation system currently in place in medical education already does this. Using a competency-based model, we discuss experiences with MLEs in other countries and the features of the examinations proposed in Brazil to measure the aptitudes of newly qualified doctors. It is concluded that, in isolation, both alternatives are inadequate. We propose the development of a broader assessment program and stress the importance of collaboration between the different actors involved in order to enhance the quality of the examinations proposed to test the aptitude of newly qualified doctors in Brazil.(AU)

La sociedad brasileña ha discutido recientemente la idea de implementar un examen de licencia (ELM) para médicos recién graduados en el país. Para evaluar la adecuación de esta propuesta y su real necesidad, se analiza en este trabajo si los ELMs establecen eficazmente la aptitud de los recién graduados para ejercer la Medicina y si los sistemas de evaluación vigentes en la enseñanza médica brasileña ya lo hacen. Usando el modelo basado en competencias como factor referencial, se discute la experiencia de la comunidad internacional con los ELMs y las características de las evaluaciones que se proponen medir, en Brasil, la aptitud de los recién graduados. Se concluye que, aisladamente, ambas alternativas son ineficaces para cumplir ese objetivo. Se propone el desarrollo de un programa más amplio de evaluación y se enfatiza la importancia de la colaboración entre los actores envueltos para perfeccionar la calidad de las evaluaciones de los recién graduados en Brasil.(AU)

Research and practice on strengthening the integration of pathophysiology teaching and national medical licensing examination / 中华医学教育探索杂志

The National Medical Licensing Examination has become one of the most important indicator s to measure the teaching quality of medical colleges and universities. In this paper, by analyzing the status of pathophysiology in National Medical Licensing Examination and the current problems existing in pathophysiology teaching, the author proposed a scheme of reform in the pathophysiology teaching based on Medical Licensing Examination, including changing teaching idea, optimizing teaching content, reforming teaching means and adjusting the assessment methods . This reform aims to make the pathophysiology teaching really serve the needs of clinical application.

Comparison of results between modified-Angoff and bookmark methods for estimating cut score of the Korean medical licensing examination / 한국의학교육

PURPOSE: The purpose of this study was to apply alternative standard setting methods for the Korean Medical Licensing Examination (KMLE), a criterion-referenced written examination, and to compare them to the conventional cut score used on the KMLE. METHODS: The process and results of criterion-referenced standard settings (i.e., the modified-Angoff and bookmark methods) were evaluated. The ratio of passing and failing examinees determined using these alternative standard setting methods was compared to the results of the conventional criteria. Additionally, the external, internal and procedural evaluation of these methods were reviewed. RESULTS: The modified-Angoff method yielded the highest cut score, followed sequentially by the conventional method and the bookmark method. The classification agreement between the modified-Angoff and bookmark methods was 0.720 measured by Cohen's κ coefficient. The intra-panelist classification consistency of modified-Angoff method was higher than bookmark method. However, the inter-panelist classification consistency was vice versa. The standard setting panelists' survey results showed that the procedures of both methods were satisfactory, but panelists had more confidence in the results of the modified-Angoff method. CONCLUSION: The modified-Angoff method showed results that were more similar to those of the conventional method. Both new methods showed very high concordance with the conventional method, as well as with each other. The modified-Angoff method was considered feasible for adoption on the KMLE. The standard setting panelists responded positively to the modified-Angoff method in terms of its practical applicability, despite certain advantages of the bookmark method.

Linear programming method to construct equated item sets for the implementation of periodical computer-based testing for the Korean Medical Licensing Examination / 보건의료교육평가

PURPOSE: This study aimed to identify the best way of developing equivalent item sets and to propose a stable and effective management plan for periodical licensing examinations. METHODS: Five pre-equated item sets were developed based on the predicted correct answer rate of each item using linear programming. These pre-equated item sets were compared to the ones that were developed with a random item selection method based on the actual correct answer rate (ACAR) and difficulty from item response theory (IRT). The results with and without common items were also compared in the same way. ACAR and the IRT difficulty were used to determine whether there was a significant difference between the pre-equating conditions. RESULTS: There was a statistically significant difference in IRT difficulty among the results from different pre-equated conditions. The predicted correct answer rate was divided using 2 or 3 difficulty categories, and the ACAR and IRT difficulty parameters of the 5 item sets were equally constructed. Comparing the item set conditions with and without common items, including common items did not make a significant contribution to the equating of the 5 item sets. CONCLUSION: This study suggested that the linear programming method is applicable to construct equated-item sets that reflect each content area. The suggested best method to construct equated item sets is to divide the predicted correct answer rate using 2 or 3 difficulty categories, regardless of common items. If pre-equated item sets are required to construct a test based on the actual data, several methods should be considered by simulation studies to determine which is optimal before administering a real test.

Thinking on the reform of clinical medicine teaching based on the national medical licensing examination / 中华医学教育探索杂志

Objective To summarize and analyze the results of our university graduates' participation in the national medical licensing examination in order to investigate the clinical medical teaching reform.Methods There were 770 candidates who took the NMLE from 2011 to 2013.Excel 2007 was used to statistically analyze the total score,passing rate,average score and mastery rate for each subject,and average score and mastery rate at different cognitive level.Results In these three years,our candidates' total pass rate was higher than the national average total pass rate,respectively higher 17.11％,18.10％,17.82％;the pass rate of practical skills examination was also higher than the national average pass rate,respectively higher 7.30％ and 7.19％ and 5.90％;the pass rate of theory examination was higher than the national average pass rate,respectively higher 12.91％,12.97％ and 14.29％.The average score and mastery rate for each subject,the average score and mastery rate at different cognitive level were also all higher than those of the national average of medical colleges.Conclusions National medical licensure examination can be used as an important reference,through optimization of medical curriculum and teaching mode,optimization of clinical practice skills training system and optimization of clinical practical skill evaluation system etc.,to further improve the clinical medical personnel training mode.

The research in the changes of design in the history taking exam of Phased Empirical Medical Licensing Examination / 中华医学教育探索杂志

History taking capability is a basic skill of clinician.During the Phased Empirical Research,NMEC redesigned the test station of History taking.On the basis of the same inquiry,we use SP,enlarge the question amount,score value,exam time,and also add the evaluation of comprehensive quality.By the new test,we can measure the history taking skill,communication talent and medical humanity competence of medical students.This paper raises a claim on humanistic quality education of medical students in order to prompt medical education to attach importance to the cultivation of medical humanity competence,to improve the medical humanistic education model,then improve the overall quality of medical students.

An empirical study on the criterion-related validity of the National Medical Licensing Exami-nation / 中华医学教育探索杂志

Objective The empirical validity of the National Medical Licensing Examination (NMLE) has rarely been studied. This is one of the first studies comparing NMLE and annual assess-ment scores. This study provided the criterion-related validity evidence for the NMLE. The NMLE can be used to test the abilities and skills of physicians. Method We evaluated the test 163 residents in 12 medical school-affiliated hospitals who had finished their first year of tralning and had taken the NMLE and an annual skills assessment. We used Pearson correlations to estimate the relationship be-tween NMLE scores and annual assessment scores. Results A total of 163 residents met inclusion requirements. Correlations between NMLE scores and annual assessment scores ranged from 0.300 to 0.843. Conclusions The criterion-related validity of NMLE was high, so it can be use to evaluate the residents clinical competence indeed.

Medical licensing examination (uigwa) and the world of the physician officers (uigwan) in Korea's Joseon Dynasty / 보건의료교육평가

Physicians for ordinary people in Joseon Dynasty (1392-1910) do not need to pass national medical licensing examination. They had done their job after enough period of apprenticeship. Only physician officers were licensed as technical civil servants. These physician officers were middle class, located socially between the nobility and the commoner. They had to pass a national licensing examination to be considered for high-ranking physician officers, that is, those at the rank above the 6th level out of a total of 9 ranks, where the first rank was highest. Royal physicians also had to pass this examination before accepting responsibility for the King's healthcare. This article aims to describe the world of Physician officers during the Joseon Dynasty. Physician officers enjoyed considerable social status because they dealt with matters of life and death. Owing to the professional nature of their fields and a strong sense of group identity they came to compose a distinct social class. The physician officers' world was marked by strong group allegiances based on shared professional knowledge; the use of marriage to gain and maintain social status; and the establishment of hereditary technical posts within the medical profession that were handed down from one generation to the next. The medical licensing examination persisted until 1894 when the civil service examination agency, of which it was part, was abolished. Until that time, the testing agency, the number of candidates who were accepted, two-step test procedures, and the method of test item selection were maintained and enforced. These aspects of the test could be considered characteristic of the medical licensing examination.

Perceptions on item disclosure for the Korean medical licensing examination / 한국의학교육

PURPOSE: This study analyzed the perceptions of medical students and faculty regarding disclosure of test items on the Korean medical licensing examination. METHODS: I conducted a survey of medical students from medical colleges and professional medical schools nationwide. Responses were analyzed from 718 participants as well as 69 faculty members who participated in creating the medical licensing examination item sets. Data were analyzed using descriptive statistics and the chi-square test. RESULTS: It is important to maintain test quality and to keep the test items unavailable to the public. There are also concerns among students that disclosure of test items would prompt increasing difficulty of test items (48.3%). Further, few students found it desirable to disclose test items regardless of any considerations (28.5%). The professors, who had experience in designing the test items, also expressed their opposition to test item disclosure (60.9%). CONCLUSION: It is desirable not to disclose the test items of the Korean medical licensing examination to the public on the condition that students are provided with a sufficient amount of information regarding the examination. This is so that the exam can appropriately identify candidates with the required qualifications.

Reflection on medical education reform from medical licensing examination / 中华医学教育探索杂志

Objective To find out the weak points of students through analyzing the scores of medical licensing examination, to explore the causes and to put forwards some advices. Methods Excel 2007 was used to statistically analyze the average scores, average passing rate and the mastering rate of 3 060 candidates who took the exam from 2006 to 2011. Literatures were reviewed to find out reforming measures taken by other schools. Results The average scores changed lightly. The master-ing rate changed differently. The average passing rate declined from 83.6% in 2006 to 71.65% in 2011. Conclusions Score of medical licensing examination is an important index of the teaching quality of medical universities, which should be emphasized on. Some practicable methods are put forward taken into all related factors:adjusting course setting, strengthening teaching of practical skills, reforming teaching methods and examination systems, placing more emphasis on examination work.

Clinical skills evaluation of the United States Medical Licensing Examinatio / 医学教育

・We visited the National Board of Medical Examiners and the Clinical Skills Evaluation Collaboration Center to discuss with the examiners the present state of the USMLE （United States Medical Licensing Examination）, to which clinical skills evaluation has been introduced.<br>・Evidence that the introduction of clinical skills evaluation to the USMLE has affected the reform of medical schools curricula in the United States supports the necessity of introducing clinical skills evaluation to the Medical Board Examination of Japan.

Improvement in Clinical Performance of Interns and Residents through Clinical Skills Assessment of the Korean Medical Licensing Examination / 한국의학교육

PURPOSE: To evaluate the clinical performance through the Korean Medical Licensing Examination clinical skills assessment (KMLE CSA) this survey was done. METHODS: A survey of 130 interns and residents (46 applicants and 84 non-applicants for the KMLE CSA) at a university hospital in Seoul was conducted in January and February 2012. The data were gathered using a structured and self-administered questionnaire. For the items that assessed the clinical performance of these subjects, we selected 15 items that are mostly frequently used by Delphi's technique, and difficult procedural skills based on the results of medical students' performance. We also used subcomponents of the clinical problems test of the KMLE CSA. RESULTS: The total score on the KMLE CSA improved by 1.33 points (a perfect score is 10), 1.49 points for procedural skills, and 0.84 points for clinical problems by multiple regression analysis. The variables that influenced clinical skills were sex (females had 0.86 more points than males), experience in military or public services (1.04 points higher than persons without experience), and type of school (graduates of medical school had 1.41 more points than graduates of professional graduate school). CONCLUSION: Implementation of the KMLE CSA improved the clinical performance of medical graduates.

The application of Angoff method in setting a standard cutoff score for comprehensive Written Test of China National Medical Licensing Examination / 中华医学教育探索杂志

This paper gives a brief introduction to China National Medical Licen sing Examination(NMLE)and focuses on the definition,modification and application protocol of Angoff Method in setting a standard for NMLE Comprehensive Written Test.Statistic analysis indicates the validity of modifled Angoff Method.Suggestion is provided for future improvement.

Introduction and Administration of the Clinical Skill Test of the Medical Licensing Examination, Republic of Korea (2009) / 보건의료교육평가

The first trial of the clinical skill test as part of the Korean Medical Licensing Examination was done from September 23 to December 1, 2009, in the clinical skill test center located in the National Health Personnel Licensing Examination Board (NHPLEB) building, Seoul. Korea is the first country to introduce the clinical skill test as part of the medical licensing examination in Asia. It is a report on the introduction and administration of the test. The NHPLEB launched researches on the validity of introducing the clinical skill test and on the best implementation methods in 2000. Since 2006, lists of subjects of test items for the clinical skill test has been developed. The test consisted of two types of evaluation, i.e., a clinical performance examination (CPX) with a standardized patient (SP) and objective structured clinical examination (OSCE). The proctor (medical faculty member) and SP rate the examinees' proficiency for the OSCE and CPX respectively. Out of 3,456 applicants, 3,289 examinees (95.2%) passed the test. Out of 167 examinees who failed the clinical skill test, 142 passed the written test. This means that the clinical skill test showed characteristics independent from the written test. This successful implementation of the clinical skill test is going to improve the medical graduates' performance of clinical skills.

Development and Changes of the United States Medical Licensing Examination(USMLE) / 中华医院管理杂志

Purpose of this study is to re-compute and analyze the hospitalization rates of residents in China and explore the methodology of systematic review in health policy research.Methodology:DThanks to a history of 90 years,USMLE has evolved into a mature system of international recognition in the industry.This paper covers the history and changes of USMLE and details its goals construction and implementation,examination question selection model,procedures and principles,selection of criteria and the development trends.

Testing Unidimensionality and Goodness-of-fitness for the Application of Item Response Theory to the Korean Medical Licensing Examination / 한국의학교육

PURPOSE: In order to apply the item response theory to test results, the assumptions made about unidimensionality and item goodness-of-fitness should be tested before analysis. To know if the Korean Medical Licensing Examination results fulfill these assumptions, appropriate tests were done on these two features. METHODS: Three results(from three subjects) of the Korean Medical Licensing Examination, done in January 2004, were converted into 1, 0 data. The Holland and Rosenbaum Method(HRMH) and DETECT were used to test unidimensionality. The Winsteps was used to test goodness-of-fitness according to the Rasch model. RESULTS: Unidimensionality was rejected by HRMH but accepted by DETECT. With the Rasch model, 2 out of 550 items were inadequate. Each 137, 73 and 455 examinees out of 3,881 were inadequate in three subjects of 126, 400 and 24 items. CONCLUSION: The above results suggest that DETECT is desirable for testing unidimensionality of high-stakes tests with more than 100 items. According to the Rasch model, the Korean Medical Licensing Examination fulfills the goodness-of-fitness to be analyzed according to the item response theory.

Enlightenment from United States medical licensing examination(USMLE) upon Chinese medical genetics education / 中华医学教育探索杂志

The article introduced the content of medical genetics in USMLE,and compared it with Chinese official teaching material. Medical genetics is an important part of USMLE and the USMLE questions come from real clinical cases and pay more attention to clinical experience. To adapt to the international trend,Chinese teaching material can clarify the genetic-related illness in detail. Clinical case is a good guide for PBL(Problem-Based learning)teaching. The medical genetics were called for in Chinese Practicing Physician Qualifications Test.

Correlation Between the Academic Score, the Trial Examination Score, and the Korean Medical Licensing Examination Score / 한국의학교육

PURPOSE: We analyzed the correlation between the year-end student academic scores (grade score), the trial examination scores, and the Korean Medical Licensing Examination (KMLE) score by correlation analysis and multiple regression method. METHODS: Our subjects included 217 graduates between the years 2002 and 2003. Yearly, trial examinations, composed according to the principles of KMLE setting, are performed four times. For correlation and regression analysis, all scores were transformed to the standard score according to the standard score norm. RESULTS: In the regression analysis among each grade scores, the lower grade score always affected the higher grade score. Also all academic grade scores correlated significantly with the trial examination scores and KMLE score in bivariate correlation analysis (P< 0.05). But only grade score during the senior years (5th and 6th) affected the trial examination and KMLE score in multiple regression. According to the multiple regressions by each grade score and trial examination score, the 6th grade score and trial exanimation score except the 1st trial examination score have a significant effect on the KMLE score. And we can also establish the regression formula such as [KMLE score]=11.377+0.752*[mean of trail examination] with significant high power of explanation (R (2) =0.709, P< 0.001). CONCLUSION: Our results show that previous scores always affect the latter scores; and the last score in time sequence is a result of the accumulation of previous education and learning. From this point of view, the trial examination is a useful tool for not only the final assessment of medical achievements but also for preparing for the KMLE.