An application of the Rasch model to reading comprehension measurement

An effective reading comprehension measurement demands robust psychometric tools that allow teachers and researchers to evaluate the educational practices and track changes in students' performance. In this study, we illustrate how Rasch model can be used to attend such demands and improve reading comprehension measurement. We discuss the construction of two reading comprehension tests: TRC-n, with narrative texts, and TRC-e, with expository texts. Three vertically scaled forms were generated for each test (TRC-n-2, TRC-n-3, TRC-n-4; TRC-e-2, TRC-e-3 and TRC-e-4), each meant to assess Portuguese students in second, third and fourth grade of elementary school. The tests were constructed according to a nonequivalent groups with anchor test design and data were analyzed using the Rasch model. The results provided evidence for good psychometric qualities for each test form, including unidimensionality and local independence and adequate reliability. A critical view of this study and future researches are discussed. (AU)

Listening Comprehension Assessment: Validity Studies of Two Vertically Scaled Tests for Portuguese Students / Evaluación de la Comprensión del Lenguaje Oral: Estudios de Validez de Dos Tests con Escalamiento Común para Alumnos Portugueses

This study aims to gather construct- and criterion-related validity evidence for the Test of Listening Comprehension of Narrative Texts (TLC-n) and the Test of Listening Comprehension of Expository Texts (TLC-e), each composed of a vertically scaled form for Portuguese students from the first to fourth grades of primary school. A sample of 1342 students and a sample of 1168 students participated in the study of the TLC-n and TLC-e forms' dimensionality, respectively. Two hundred and eighty students participated in the criterion-related validity study of the tests. Confirmatory factorial analyses were performed for the study of the test forms' dimensionality. To collect criterion-related validity evidence, Pearson correlations with measures oflistening and reading related skills were calculated. Results provide evidence of an acceptable fit for the one-factor solution, for all forms, and statistically significant correlations between the test forms and the external criteria measures. There is evidence for the validity of the test forms.

Los objetivos de este trabajo fueron recoger evidencia de validez de constructo y de criterio para dos tests, el Test of Listening Comprehension of Narrative Texts (TLC-n) y el Test of listening Comprehension of Expository Texts (TLC-e), con cuatro versiones escaladas verticalmente para evaluar la comprensión oral de los alumnos portugueses de enseñanza primaria. Una muestra de 1342 alumnos y una muestra de 1168 alumnos participaron en el estudio de la dimensionalidad del TLC-n y TLC-e. Participaron 260 alumnos en el estudio de validez de criterio de los tests. La dimensionalidad de los tests se examinó mediante un procedimiento de análisis factorial confirmatorio. Para recoger datos sobre evidencia de validez de criterio, se calcularon las correlaciones de Pearson con otras pruebas de comprensión oral y de lectura. Los resultados revelaron que el modelo de un factor se ajusta a los datos. Se obtuvieron coeficientes de correlación estadísticamente significativos entre el TLC-n y TLC-e y los otros tests. Los resultados apoyan la validez de las versiones de los tests.