ABSTRACT
Objective To evaluate the reliability of objective structured clinical examination based on standardized patients in clinical skill test for medical students.Methods To analyze the scores of 8 stations OSCE by applying multivariate generalizability theory.Results The relative G coefficient of the 8 stations OSCE was 0.886, and the absolute G coefficient was 0.883.Conclusions Multivariate generalizability theory is an objective and scientific method to evaluate the OSCE stations, and the analysis results facilitate to improvement the practice interms of quality and efficacy.
ABSTRACT
Objective To analysis the reliability of the examination of Introduction of Nursing by using the methods of generalizability theory and to give advice on how to modify examination paper.Methods Estimation value of variance component from different variations and precision indexes were calculated by applying the principle of generalizability theory.Results The generalizability coefficient and index of dependability were greater than 0.4 in choice question and terminology explanation,the generalizability coefficient of short-answer question was also greater than 0.4.Short-answer question's index of dependability and generalizability coefficient and index of dependability of comprehensive problem were less than 0.4.Conclusions Generalizability theory can be used in analysis the reliability of the examination of Introduction of Nursing.Its result is helpful to improve the framework of examination papers and the reliability of examination.
ABSTRACT
Objective To systematically evaluate the overall reliability,test paper structure and rationality proportional distribution of the secondary discipline of western medicine comprehensive test papers from 2010 to 2012 to provide references for future improvement to the syllabus and test quality. Methods Data of 15 765 candidates from 2010 to 2012 were extracted and statistical analy-sis was performed to do generalized research and decision-making research from discipline factors and question type factors. Results ①Based on the discipline factor analysis,variance components of ba-sic disciplines were relatively high ,global score relative error was very small and the generalizability coefficient was above 0.930. ②Based on the question type factor analysis,variance components in X-type questions and A1-type questions were relatively high, global score relative error was very small and the generalizability coefficient was above 0.930. Conclusions ①From the discipline content, the accuracy of various disciplines is high;contribution of basic disciplines is relatively higher than that of clinical disciplines;further improving the discrimination of questions can help to improve measure-ment accuracy. ②From the question design,question measurement error is small,accuracy is high,and the question distribution is appropriate;scale of A3 type questions can be further improved.
ABSTRACT
Objective: To evaluate the reliability of QLICP-GA (quality of life instruments for cancer patients-gastric cancer) scale by using generalizability theory, and to provide the references in order to revise and improve this scale. Methods: The mean squares of person (p), item (i) and the person-by-item interaction effect (pi) were calculated by using the module of UNIANOVA under the menu of General Linear Model in SPSS software. Furthermore, the generalizability coefficients and indexes of dependability were calculated in order to assess the reliability of QLICP-GA scale. Results: The largest source of variation in every domain was due to person-by-item interaction. The variances accounted for by person were large, while the variation accounted for by item was minimal. Generalizability coefficients of five domains were 0.774 (physical domain), 0.854 (psychological domain), 0.670 (social domain), 0.717 (common symptom and side effect domain), and 0.696 (specific domain), respectively. Conclusion: Physical domain, psychological domain and common symptom and side effect domain in QLICP-GA have good reliability. However, social domain and specific domain have acceptable reliability. Social domain should be emphasized when this scale will be revised and improved in future. The number of items of social domain should be added from 6 to 8 or 10. Secondly, the number of items of specific domain may be slightly added from 7 to 9. Copyright © 2013 by TUMOR.
ABSTRACT
PURPOSE: The purpose of this study is to test the reliability of the clinical performance examination (CPX) using Generalizability theory (G-theory). Through G-theory, the effects of not only students and tasks but also the school will be analyzed as primary sources of error, which can affect the interpretation of the reliability of the CPX. METHODS: One thousand three hundred nineteen students from 16 medical schools that participated in the Seoul-Gyeonggi CPX Consortium 2008 were enrolled. In our research design, we suppose that student is nested within school and crossed with task. Data analysis was conducted with urGenova. RESULTS: According to our analysis, the percentage of error variance was 6.2% for school, 14.9% for student nested within school, 14.4% for task, and 3% for interaction between school and task. An effect of school on students was observed, but the interaction between task and school was insignificant. When student is nested within school, the universe score decreased and the g-coefficient was less than the g-coefficient of the p x t (p: studentm, t: task) design. CONCLUSION: The results show that generalizability theory is useful in detecting various error components in the CPX. Using the generalizability theory to improve the technical quality of performance assessments provides us with greater information compared with traditional test theories.