Enhancing Content Validity Assessment With Item Response Theory Modeling / Mejorando la Evaluación de la Validez del Contenido a Través del Modelado de la Teoría de Respuesta al Ítem

Background: Ensuring the validity of assessments requires a thorough examination of the test content. Subject matter experts (SMEs) are commonly employed to evaluate the relevance, representativeness, and appropriateness of the items. This article proposes incorporating item response theory (IRT) into model assessments conducted by SMEs. Using IRT allows for the estimation of discrimination and threshold parameters for each SME, providing evidence of their performance in differentiating relevant from irrelevant items, thus facilitating the detection of suboptimal SME performance while improving item relevance scores. Method: Use of IRT was compared to traditional validity indices (content validity index and Aiken’s V) in the evaluation of conscientiousness items. The aim was to assess the SMEs’ accuracy in identifying whether items were designed to measure conscientiousness or not, and predicting their factor loadings. Results: The IRT-based scores effectively identified conscientiousness items (R2 = 0.57) and accurately predicted their factor loadings (R2 = 0.45). These scores demonstrated incremental validity, explaining 11% more variance than Aiken’s V and up to 17% more than the content validity index. Conclusions: Modeling SME assessments with IRT improves item alignment and provides better predictions of factor loadings, enabling improvement of the content validity of measurement instruments.(AU)

Antecedentes: Garantizar la validez de evaluaciones requiere un examen exhaustivo del contenido de una prueba. Es común emplear expertos en la materia (EM) para evaluar la relevancia, representatividad y adecuación de los ítems. Este artículo propone integrar la teoría de respuesta al ítem (TRI) en las evaluaciones hechas por EM. La TRI ofrece parámetros de discriminación y umbral de los EM, evidenciando su desempeño al diferenciar ítems relevantes/ irrelevantes, detectando desempeños subóptimos, mejorando también la estimación de la relevancia de los ítems. Método: Se comparó el uso de la TRI frente a índices tradicionales (índice de validez de contenido y V de Aiken) en ítems de responsabilidad. Se evaluó la precisión de los EM al discriminar si los ítems medían responsabilidad o no, y si sus evaluaciones permitían predecir los pesos factoriales de los ítems. Resultados: Las puntuaciones de TRI identificaron bien los ítems de responsabilidad (R2 = 0,57) y predijeron sus cargas factoriales (R2 = 0,45). Además, mostraron validez incremental, explicando entre 11% y 17% más de varianza que los índices tradicionales. Conclusiones: La TRI en las evaluaciones de los EM mejora la alineación de ítems y predice mejor los pesos factoriales, mejorando validez del contenido de los instrumentos.(AU)

Factor Convergence and Predictive Analysis of the Five Factor and Alternative Five Factor Personality Models with the Five-Factor Personality Inventory for ICD-11 (FFICD).

The authors analyze and compare the factor convergence and predictive power of the Revised NEO Personality Inventory (NEO-PI-R) and the Zuckerman-Kuhlman-Aluja Personality Questionnaire (ZKA-PQ/SF) with respect to the Five-Factor Personality Inventory for ICD-11 (FFiCD). A total of 803 White Spanish subjects were analyzed. All the personality domains had significant predictive power with regard to the FFiCD except NEO Openness. The explained variance of the personality domains with respect to FFiCD Negative Affectivity (71% and 77%) and Detachment (56% and 56%) were similar for NEO-PI-R and ZKA-PQ/SF, respectively, but the NEO-PI-R accounted for greater variance for FFiCD Anankastia, Dissociality, and Disinhibition. The FFiCD facets of Rashness, Thrill- Seeking (Disinhibition), and Unassertiveness (Detachment) were located in factors other than those theoretically expected. The authors conclude that normal personality measured by the NEO-PI-R and the ZKA-PQ/SF contribute, in a differential but complementary way, to knowledge of the maladaptive personality measured by the FFiCD.

Factor Convergence and Predictive Analysis of the Five Factor and Alternative Five Factor Personality Models With the Five-Factor Personality Inventory for ICD-11 (FFiCD).

The authors analyze and compare the factor convergence and predictive power of the Revised NEO Personality Inventory (NEO-PI-R) and the Zuckerman-Kuhlman-Aluja Personality Questionnaire (ZKA-PQ/SF) with respect to the Five-Factor Personality Inventory for ICD-11 (FFiCD). A total of 803 White Spanish subjects were analyzed. All the personality domains had significant predictive power with regard to the FFiCD except NEO Openness. The explained variance of the personality domains with respect to FFiCD Negative Affectivity (71% and 77%) and Detachment (56% and 56%) were similar for NEO-PI-R and ZKA-PQ/SF, respectively, but the NEO-PI-R accounted for greater variance for FFiCD Anankastia, Dissociality, and Disinhibition. The FFiCD facets of Rashness, Thrill- Seeking (Disinhibition), and Unassertiveness (Detachment) were located in factors other than those theoretically expected. The authors conclude that normal personality measured by the NEO-PI-R and the ZKA-PQ/SF contribute, in a differential but complementary way, to knowledge of the maladaptive personality measured by the FFiCD.