ABSTRACT
Objective:To test the reliability of the short version of the depression-anxiety-stress scale (DASS-21) in adolescents through generalizability theory.Methods:Total of 458 adolescents from three middle schools in Guizhou and Sichuan province were surveyed with the DASS-21 from November to December 2021.Based on multivariate generalization theory(MGT), the generalization coefficients, reliability indexes and variance contribution ratios were calculated.mGENOVA 2.1 was used for data analysis.Results:The generalizability coefficient and reliability index for the total scale of DASS-21 were 0.927 and 0.922, respectively.The generalizability coefficients of depression, anxiety and stress subscales were 0.808-0.840, and the dependability indexes were 0.789-0.832.The contribution ratios of depression, anxiety and stress subscales to the general score were 33.20%, 33.94% and 32.86%, respectively.The stress subscale (34.95%) contributed the most to the relative error variance, and the anxiety subscale (36.98%) contributed the most to the absolute error variance.Conclusion:The DASS-21 has excellent test reliability in adolescents, and it can be used both as the norm-referenced test and the criterion-referenced test in the actual assessment of negative emotions in adolescents.
ABSTRACT
This paper describes test theory, which is the theoretical foundation of learner assessment, in relation to its application in Common Achievement Tests. To be specific, classical test theory and the reliability coefficients, generalizability theory, and item response theory are taken up. In classical test theory, the observed score X is expressed as the sum of a true-score component T and an error component E. The ratio of the true-score variance to the observed score variance defines the reliability coefficient. Generalizability theory is an extension of the notion of reliability in classical test theory. Item response theory overcomes the limitations of classical test theory and can express the properties of items (difficulty and discrimination) and examinees’ abilities separately.
ABSTRACT
Radiomics, which involves the use of high-dimensional quantitative imaging features for predictive purposes, is a powerful tool for developing and testing medical hypotheses. Radiologic and statistical challenges in radiomics include those related to the reproducibility of imaging data, control of overfitting due to high dimensionality, and the generalizability of modeling. The aims of this review article are to clarify the distinctions between radiomics features and other omics and imaging data, to describe the challenges and potential strategies in reproducibility and feature selection, and to reveal the epidemiological background of modeling, thereby facilitating and promoting more reproducible and generalizable radiomics research.
Subject(s)
Machine LearningABSTRACT
Objective To evaluate the reliability of objective structured clinical examination based on standardized patients in clinical skill test for medical students.Methods To analyze the scores of 8 stations OSCE by applying multivariate generalizability theory.Results The relative G coefficient of the 8 stations OSCE was 0.886, and the absolute G coefficient was 0.883.Conclusions Multivariate generalizability theory is an objective and scientific method to evaluate the OSCE stations, and the analysis results facilitate to improvement the practice interms of quality and efficacy.
ABSTRACT
El propósito de este trabajo fue optimizar una nueva herramienta de evaluación de la calidad de servicio percibida en los Centros de Atención Temprana, desarrollada ad hoc, que permitiera analizar distintas dimensiones desde la perspectiva de la Teoría de la Generalizabilidad. El objetivo fue desglosar la variabilidad real de la variabilidad del error. Se estimaron los componentes de varianza de las facetas centros, usuarios, ítems y escalas. Se utilizó una estructura de diseño multifaceta parcialmente anidado, analizando distintas categorías independientemente y en interacción, obteniendo excelentes resultados tanto de los índices de fiabilidad como de generalizabilidad. Se realizó una optimización del diseño de medida para optimizar los tamaños muestrales, pudiendo ser considerado un análisis coste-beneficio.
The purpose of this work was to optimize a new tool for evaluation of quality perceived in the Early Intervention Centers, developed ad hoc, allowing to analyze different dimensions of quality service in these kind of centres from the perspective of the Generalizability Theory. The aim was to break down the actual variability of the variability of the error. The variance of the facets centers, users, items and scales components were estimated. We used a structure of multifaceted partially nested design, analyzing different categories independently and in interaction, obtaining excellent results both indexes of reliability and generalizability. Measure design optimization to optimize sample sizes, and can be considered a cost-benefit analysis was conducted.
Subject(s)
Quality Assurance, Health Care , Child DevelopmentABSTRACT
Objective To analysis the reliability of the examination of Introduction of Nursing by using the methods of generalizability theory and to give advice on how to modify examination paper.Methods Estimation value of variance component from different variations and precision indexes were calculated by applying the principle of generalizability theory.Results The generalizability coefficient and index of dependability were greater than 0.4 in choice question and terminology explanation,the generalizability coefficient of short-answer question was also greater than 0.4.Short-answer question's index of dependability and generalizability coefficient and index of dependability of comprehensive problem were less than 0.4.Conclusions Generalizability theory can be used in analysis the reliability of the examination of Introduction of Nursing.Its result is helpful to improve the framework of examination papers and the reliability of examination.
ABSTRACT
Objective To systematically evaluate the overall reliability,test paper structure and rationality proportional distribution of the secondary discipline of western medicine comprehensive test papers from 2010 to 2012 to provide references for future improvement to the syllabus and test quality. Methods Data of 15 765 candidates from 2010 to 2012 were extracted and statistical analy-sis was performed to do generalized research and decision-making research from discipline factors and question type factors. Results ①Based on the discipline factor analysis,variance components of ba-sic disciplines were relatively high ,global score relative error was very small and the generalizability coefficient was above 0.930. ②Based on the question type factor analysis,variance components in X-type questions and A1-type questions were relatively high, global score relative error was very small and the generalizability coefficient was above 0.930. Conclusions ①From the discipline content, the accuracy of various disciplines is high;contribution of basic disciplines is relatively higher than that of clinical disciplines;further improving the discrimination of questions can help to improve measure-ment accuracy. ②From the question design,question measurement error is small,accuracy is high,and the question distribution is appropriate;scale of A3 type questions can be further improved.
ABSTRACT
Objective: To evaluate the reliability of QLICP-GA (quality of life instruments for cancer patients-gastric cancer) scale by using generalizability theory, and to provide the references in order to revise and improve this scale. Methods: The mean squares of person (p), item (i) and the person-by-item interaction effect (pi) were calculated by using the module of UNIANOVA under the menu of General Linear Model in SPSS software. Furthermore, the generalizability coefficients and indexes of dependability were calculated in order to assess the reliability of QLICP-GA scale. Results: The largest source of variation in every domain was due to person-by-item interaction. The variances accounted for by person were large, while the variation accounted for by item was minimal. Generalizability coefficients of five domains were 0.774 (physical domain), 0.854 (psychological domain), 0.670 (social domain), 0.717 (common symptom and side effect domain), and 0.696 (specific domain), respectively. Conclusion: Physical domain, psychological domain and common symptom and side effect domain in QLICP-GA have good reliability. However, social domain and specific domain have acceptable reliability. Social domain should be emphasized when this scale will be revised and improved in future. The number of items of social domain should be added from 6 to 8 or 10. Secondly, the number of items of specific domain may be slightly added from 7 to 9. Copyright © 2013 by TUMOR.
ABSTRACT
PURPOSE: The purpose of this study is to test the reliability of the clinical performance examination (CPX) using Generalizability theory (G-theory). Through G-theory, the effects of not only students and tasks but also the school will be analyzed as primary sources of error, which can affect the interpretation of the reliability of the CPX. METHODS: One thousand three hundred nineteen students from 16 medical schools that participated in the Seoul-Gyeonggi CPX Consortium 2008 were enrolled. In our research design, we suppose that student is nested within school and crossed with task. Data analysis was conducted with urGenova. RESULTS: According to our analysis, the percentage of error variance was 6.2% for school, 14.9% for student nested within school, 14.4% for task, and 3% for interaction between school and task. An effect of school on students was observed, but the interaction between task and school was insignificant. When student is nested within school, the universe score decreased and the g-coefficient was less than the g-coefficient of the p x t (p: studentm, t: task) design. CONCLUSION: The results show that generalizability theory is useful in detecting various error components in the CPX. Using the generalizability theory to improve the technical quality of performance assessments provides us with greater information compared with traditional test theories.
Subject(s)
Humans , Research Design , Schools, Medical , Statistics as TopicABSTRACT
PURPOSE: Selecting medical students through interviews seems difficult and the reliability of the results is one of the major concerns. The purpose of this study was to investigate the reliability and acceptability of the Multiple Mini-Interview (MMI) in selecting medical students of Kangwon National University. METHODS: Eighty-four applicants participated in the MMI which consists of 3 8-minute stations that have 9 checklist items and 3 global items. The 3 domains that we chose were motivation to become a doctor, communication and interpersonal skills, and ethical decision-making. We placed 2 interviewers in each room. The interviewers were chosen from our faculty. We analyzed the reliability of the MMI with urGENOVA for PC. We conducted a survey of these applicants and interviewers. RESULTS: The reliability was 0.791. Students answered that the interview was impressive and enjoyable. Students were also satisfied with the level and quality of the MMI cases. They described that they were evaluated objectively. Interviewers also responded positively. They stated that more stations and more efforts to develop the cases were needed to improve the reliability and validity. CONCLUSION: The MMI was acceptable to our applicants and faculty. It is reliable for assessing medical school applicants in Korea. We should develop more stations and better cases to increase the reliability and validity of the MMI.