RESUMO
En el marco de las políticas públicas en Colombia sobre calidad de la educación superior se encuentran los lineamientos para que el Estado reconozca los programas de pregrado como de alta calidad, entre tales lineamientos se consideran los sistemas de evaluación de los estudiantes basados en políticas institucionales claras y transparentes que propicien una identificación integral de las condiciones académicas y actitudinales de los estudiantes por lo que es imprescindible tecnificar y modernizar los procesos de evaluación para fortalecer la emisión de inferencias confiables sobre el logro académico. En este artículo se describe una aplicación de la Teoría de Grafos usando algunas de sus definiciones y teoremas básicos con la intención de potenciar su uso para el estudio del supuesto de unidimensionalidad de la Teoría de Respuesta al Ítem como alternativa para robustecer la evaluación. La unidimensional fue entendida en el sentido de la homogeneidad escalar y de criterio para las relaciones de orden definidas por la dominación tanto para ítems como para evaluados: para los ítems conforme al número de evaluados que los contestaron correctamente y a la dificultad; para los evaluados respecto al número de respuestas correctas y a la habilidad. Se aplicó la combinación de las dos teorías a un caso de estudio que contempla el instrumento de la prueba diagnóstica del Examen de Clasificación de Matemáticas Básicas en la Universidad Jorge Tadeo Lozano, con esto se ofrece un valor agregado porque se evidencian los resultados de forma algorítmica y visual con ayuda de herramientas informáticas especializadas.
In Colombia, public policies about quality of higher education involve student assessment systems as a relevant characteristic. These systems should have clear and transparent institutional policies requiring comprehensive identification of attitudinal and academic conditions about examinee performance. Consequently, on the way of continuous improvement of academic services offered by institutions of higher education, an assessment process adequately technified allowing valid inferences about academic achievement is required. In this sense, the Rasch measurement model of Item Response Theory is a modern alternative to strengthen educational measurement estimating the ability of the student and the difficulty of the item on a comparable scale. Unidimensionality, local independence and internal consistency are assumptions made in Rasch measurement model. Unidimensionality assumption has several definitions, one of them is the occurrence of a dominant factor influencing test performance. Another definition is considered in the present study. Here, unidimensionality was interpreted as scalar and criterial homogeneity for the order relations defined by domination for items and assessed applicants. Graph Theory is an ideal mathematical modelling approach to this assumption inasmuch as represent intangible interactions as required. In order to achieve this, Graph Theory and Item Response Theory were combined to examine the qualifying test for the Basic Mathematics subject at Jorge Tadeo Lozano University as a case study. This test was composed by 45 items divided in three blocks. With several combinations of these blocks, three virtual booklets were obtained each one comprising 30 of them. From the application of June 2011, a test sample of 509 responses chains was obtained. Three data bases were processed one for each booklet, the first spanned 175 responses chains, the second 170 and the third one 164. To begin with the exmination on the Rasch measurement model, the parameters reliability, separation, Cronbach Alpha and item residual correlation were estimated to gauge and determine performance test for getting acceptable values in each booklet applied. Tatsuoka was followed to undertake the path to unidimensionality from Graph Theory gathering a real case experience processed with suitable software. Additionally, the sensitivity of the order relation was verified through: 1) ordering by number of correct responses per item (1I order) and items difficulty (2I order); 2) ordering by number of correct responses per applicant (1E order) and estimated ability (2E order). Furthermore, the linear models were obtained collating these orders. In like manner, three Guttman scales and their adjacency matrices were schematized one for each booklet. Subsequently, the respective graphs were processed and represented using Gephi just as a specialized tool that enables running some algorithms like Force Atlas. Afterwards, the second power of each adjacency matrix was found using Matlab 2014b and domination matrices were calculated for both items and applicants in the aforementioned orders for a total of 12 matrices. Consistency index developed by Cliff was computed for the domination matrices. As a result, moderate consistency was observed. Significant domination for the entries of these matrices was analyzed through McNemar test in order to have an asymmetric dominance relation. Moreover, a reachable matrix was calculated for each one of these significant domination matrices as a limit ofa sequence of boolean powers. Finally, dominance hierarchies were illustrated with vertex degrees and compared with student maps by the means of Winsteps 3.73. The combination of Graph Theory and Item Response Theory allowed a deeper comprehension of unidimensionality assumption. Thereupon, universities can optimize their resources offering to applicant differential academic options per individual position in the ability scale. The results can be used to outline advantages for the applicants who can evidence their position in the ability scale and identify the different areas to improve.
RESUMO
A análise fatorial exploratória pode ser utilizada para verificar a unidimensionalidade de escalas construídas para avaliar aspectos referentes a dimensões específicas da interação professor-aluno. O presente trabalho teve como objetivo demonstrar a avaliação da unidimensionalidade para as três Escalas (Responsividade, Exigência e Controle Coercitivo) do Inventário de Estilos de Liderança de Professores -IELP, utilizando-se os recursos da análise fatorial exploratória. Participaram dessa pesquisa 113 alunos de 4º e 5º anos do Ensino Fundamental, com idade entre 8 e 11 anos, provenientes de escolas pública e privada de municípios do interior do Paraná. Foi utilizado o IELP com os 70 itens. A aplicação do instrumento foi coletiva com grupos de, no máximo, 25 alunos, em salas de aula. Nos resultados foi constatada a distribuição normal dos dados e que as escalas que compõem o IELP são unidimensionais. Em função da análise das cargas fatoriais, o número de itens em cada Escala foi reduzido: na Responsividade passou de 22 para 21 itens, na Exigência passou de 22 para 16 e na Escala Controle Coercitivo, passou de 26 para 19 itens. Dessa forma, a nova versão do IELP ficou com 56 itens que apresentaram carga fatorial > 0,3.
Exploratory factor analysis can be used to verify the unidimensionality of scales constructed to evaluate aspects related to specific dimensions of the teacher-student interaction. This study aimed to demonstrate the evaluation of unidimensionality for the three scales (Responsiveness, Demand and Coercive Control) of the Teacher Leadership Styles Inventory - TLSI, using exploratory factor analysis. Participants of the study were 113 students of the 4th and 5th years of elementary education, aged between 8 and 11 years, from public and private schools of cities in the state of Paraná. The 70 item TLSI was used. The application of the instrument was performed collectively with groups of no more than 25 students, in the classrooms. The results demonstrated the normal distribution of the data and that the scales that compose the TLSI are unidimensional. From the analysis of the factor loadings, the number of items in each scale was reduced: from 22 to 21 items in the Responsiveness scale, from 22 to 16 items in the Demand scale and from 26 to 19 items in the Coercive Control scale. Thus, the new version of the TLSI contained 56 items that showed factor loadings > .3.
El análisis factorial exploratorio puede ser utilizado para averiguar la dimensionalidad de escalas construídas para evaluar aspectos relacionados a dimensiones específicas de la interacción profesor-alumno. El presente trabajo tuvo como objetivo demostrar la evaluación de la unidimensionalidad para las tres Escalas (responsividad, exigencia y control coercitivo) del Inventario de Estilos de Liderazgo de Profesores - IELP, utilizándose los recursos del análisis factorial exploratorio. Participaron de esa investigación 113 alumnos de 4º y 5º años de la Enseñanza Fundamental, con edad entre 8 y 11 años, provenientes de escuelas pública y privada de municipios del interior de Paraná. Fue utilizado el IELP con 70 artículos. La aplicación del instrumento fue colectiva con grupos de, al máximo, 25 alumnos, en salones de clase. En los resultados fue contatada la distribución normal de los datos y que las escalas que componen el IELP son unidimensionales. En función de las cargas factoriales, el número de artículos en cada Escala fue reducido: en responsividad fue de 22 para 21 artículos, en Exigencia fue de 22 para 16 y en la Escala Control Coercitivo, fue de 26 para 19 artículos. De esta manera, la versión de IELP quedó con 56 artículos que presentaron carga factorial > .3.
RESUMO
PURPOSE: In order to apply the item response theory to test results, the assumptions made about unidimensionality and item goodness-of-fitness should be tested before analysis. To know if the Korean Medical Licensing Examination results fulfill these assumptions, appropriate tests were done on these two features. METHODS: Three results(from three subjects) of the Korean Medical Licensing Examination, done in January 2004, were converted into 1, 0 data. The Holland and Rosenbaum Method(HRMH) and DETECT were used to test unidimensionality. The Winsteps was used to test goodness-of-fitness according to the Rasch model. RESULTS: Unidimensionality was rejected by HRMH but accepted by DETECT. With the Rasch model, 2 out of 550 items were inadequate. Each 137, 73 and 455 examinees out of 3,881 were inadequate in three subjects of 126, 400 and 24 items. CONCLUSION: The above results suggest that DETECT is desirable for testing unidimensionality of high-stakes tests with more than 100 items. According to the Rasch model, the Korean Medical Licensing Examination fulfills the goodness-of-fitness to be analyzed according to the item response theory.