Validity evidence based on internal structure / Evidencia de validez basada en la estructura interna

BACKGROUND: Validity evidence based on the internal structure of an assessment is one of the five forms of validity evidence stipulated in the Standards for Educational and Psychological Testing of the American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. In this paper, we describe the concepts underlying internal structure and the statistical methods for gathering and analyzing internal structure. METHOD: An in-depth description of the traditional and modern techniques for evaluating the internal structure of an assessment. RESULTS: Validity evidence based on the internal structure of an assessment is necessary for building a validity argument to support the use of a test for a particular purpose. CONCLUSIONS: The methods described in this paper provide practitioners with a variety of tools for assessing dimensionality, measurement invariance and reliability for an educational test or other types of assessment

ANTECEDENTES: la evidencia de validez basada en la estructura interna de una evaluación es una de las cinco formas de evidencias de validez estipuladas en los Standards for Educational and Psychological Testing de la American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. En este artículo describimos los conceptos que subyacen a la estructura interna y los métodos estadísticos para analizarla. MÉTODO: una descripción detallada de las técnicas tradicionales y modernas para evaluar la estructura interna de una evaluación. RESULTADOS: la evidencia de validez basada en la estructura interna de una evaluación es necesaria para elaborar un argumento de validez que apoye el uso de un test para un objetivo particular. CONCLUSIONES: los métodos descritos en este artículo aportan a los profesionales una variedad de herramientas para evaluar la dimensionalidad, invarianza de la medida y fiabilidad de un test educativo u otro tipo de evaluación

A comparison of internal model validation methods for multifactor dimensionality reduction in the case of genetic heterogeneity.

BACKGROUND: Determining the genes responsible for certain human traits can be challenging when the underlying genetic model takes a complicated form such as heterogeneity (in which different genetic models can result in the same trait) or epistasis (in which genes interact with other genes and the environment). Multifactor Dimensionality Reduction (MDR) is a widely used method that effectively detects epistasis; however, it does not perform well in the presence of heterogeneity partly due to its reliance on cross-validation for internal model validation. Cross-validation allows for only one "best" model and is therefore inadequate when more than one model could cause the same trait. We hypothesize that another internal model validation method known as a three-way split will be better at detecting heterogeneity models. RESULTS: In this study, we test this hypothesis by performing a simulation study to compare the performance of MDR to detect models of heterogeneity with the two different internal model validation techniques. We simulated a range of disease models with both main effects and gene-gene interactions with a range of effect sizes. We assessed the performance of each method using a range of definitions of power. CONCLUSIONS: Overall, the power of MDR to detect heterogeneity models was relatively poor, especially under more conservative (strict) definitions of power. While the overall power was low, our results show that the cross-validation approach greatly outperformed the three-way split approach in detecting heterogeneity. This would motivate using cross-validation with MDR in studies where heterogeneity might be present. These results also emphasize the challenge of detecting heterogeneity models and the need for further methods development.