Identifying Disengaged Responding in Multiple-Choice Items: Extending a Latent Class Item Response Model With Novel Process Data Indicators.

Disengaged responding poses a severe threat to the validity of educational large-scale assessments, because item responses from unmotivated test-takers do not reflect their actual ability. Existing identification approaches rely primarily on item response times, which bears the risk of misclassifying fast engaged or slow disengaged responses. Process data with its rich pool of additional information on the test-taking process could thus be used to improve existing identification approaches. In this study, three process data variables-text reread, item revisit, and answer change-were introduced as potential indicators of response engagement for multiple-choice items in a reading comprehension test. An extended latent class item response model for disengaged responding was developed by including the three new indicators as additional predictors of response engagement. In a sample of 1,932 German university students, the extended model indicated a better model fit than the baseline model, which included item response time as only indicator of response engagement. In the extended model, both item response time and text reread were significant predictors of response engagement. However, graphical analyses revealed no systematic differences in the item and person parameter estimation or item response classification between the models. These results suggest only a marginal improvement of the identification of disengaged responding by the new indicators. Implications of these results for future research on disengaged responding with process data are discussed.

Thriving from work questionnaire: German translation and validation.

BACKGROUND: The Thriving from Work questionnaire is a comprehensive indicator of positive well-being for employees, applicable in both research and practical contexts. Current discussions underline the crucial impact that employment should have in enriching workers' lives positively and meaningfully, along with the necessity for accurate and dependable tools to assess employee well-being. This study investigated the reliability, validity, and dimensionality of the translated German adaptation of the Thriving from Work questionnaire developed by Peters and colleagues [1, 2]. The questionnaire assesses work-related well-being with 30 items clustered in six domains: emotional and psychological well-being, social well-being, work-life integration, physical and mental well-being, basic needs for thriving, and experiences of work. METHODS: This study aimed to convert the Thriving at Work Questionnaire from English into German. We assessed the psychometric characteristics of the German version of the questionnaire by using item response theory with a sample of 567 German employees and examined its criterion validity. RESULTS: We found that the long and short German Thriving from Work questionnaire versions are reliable with good construct validity. Criterion validity was demonstrated by relationships with important work and life outcomes, such as life satisfaction, trust in the organizations' management, general well-being, work-related fatigue, and work stress. CONCLUSIONS: The current study demonstrated that the German language version of the questionnaire is both a reliable and valid measure of employee well-being. We discuss recommendations for further adaptation and future research.

Evaluation and refinement of Self-Directed Learning Readiness Scale for medical students.

PURPOSE: This study evaluated the underlying subdomain structure of the Self-Directed Learning Readiness Scale (SDLRS) for medical students and refined the instrument to measure the subdomains to provide evidence for construct validity. Developing self-directed learners is a well-recognized goal amongst medical educators. The SDLRS has been frequently used, however, lack of construct validity makes it difficult to interpret results. METHODS: To identify the valid subdomains of the SDLRS, items were calibrated with the graded response model (GRM) and results were used to construct a 30-item short form. Short-form validity was evaluated by examining the correspondence between the total scores from the short form and the original instrument for individual students. RESULTS: A five-subdomain model explained the SDLRS item response data reasonably well. These included: (1) initiative and independence in learning, (2) self-concept as an effective learner, (3) openness to learning opportunity, (4) love of learning, and (5) acceptance for one's own learning. The unidimensional GRM for each subdomain fits the data better than multi-dimensional models. The total scores from the refined short form and the original form were correlated at 0.98 and the mean difference was 1.33, providing evidence for validation. Nearly 91% of 179 respondents were accurately classified within the low, average, and high readiness groups. CONCLUSION: Sufficient evidence was obtained for the validity and reliability of the refined 30-item short-form targeting five subdomains to measure medical students' readiness to engage in self-directed learning.

PROpwr: a Shiny R application to analyze patient-reported outcomes data and estimate power.

Patient Reported Outcomes (PROs) are widely used in quality of life (QOL) studies, health outcomes research, and clinical trials. The importance of PRO has been advocated by health authorities. We propose this R shiny web application, PROpwr, that estimates power for two-arm clinical trials with PRO measures as endpoints using Item Response Theory (GRM: Graded Response Model) and simulations. PROpwr also supports the analysis of PRO data for convenience of estimating the effect size. There are seven function tabs in PROpwr: Frequentist Analysis, Bayesian Analysis, GRM power, T-test Power Given Sample Size, T-test Sample Size Given Power, Download, and References. PROpwr is user-friendly with point-and-click functions. PROpwr can assist researchers to analyze and calculate power and sample size for PRO endpoints in clinical trials without prior programming knowledge.

The relationship between ambulance team's professional commitment, occupational anxiety, and resilience levels.

BACKGROUND: Individuals who have the ability to bounce back from stressful events, to recover from their troubles and adverse environmental conditions by getting stronger each time are known as resilient people. Some professions may cause more occupational anxiety than others due to their characteristics and working conditions. In this research, we aimed to develop a professional commitment scale for the ambulance team. Another aim was to analyze the relationships between professional commitment, occupational anxiety, resilience, gender, job, seniority and working unit variables. METHODS: In the study, data were collected from a total of 1142 emergency ambulance workers working in Emergency Ambulance and Emergency Call Centers in 34 different cities in Turkey. Data were collected using the "Professional Commitment of Ambulance Team Scale (PCATS), Occupational Anxiety Scale for Emergency Medical Service Professionals (OASEMSP), and Resilience Scale for Adults (RSA). Scale development analyses were carried out using Classical Test Theory (CTT) and Item Response Theory (IRT). Regression analysis were used to examine the relationships between professional commitment, occupational anxiety, resilience, gender, job, seniority and working unit. RESULTS: As a result of the exploratory factor analysis (EFA), it was determined that 8 items remaining in the professional commitment scale formed a single-factor structure, explaining 46% of the variance of professional commitment of the team. The Cronbach's Alpha reliability value was 0.867. Confirmatory factor analysis (CFA) confirmed the results of exploratory factor analysis. The Cronbach's Alpha reliability coefficient obtained through CTT was 0.868, and the marginal reliability coefficient within the scope of IRT was 0.877. The test-retest reliability coefficient was calculated as 0.832, which indicates that the scale is valid and reliable. CONCLUSIONS: The study revealed that resilience has a positive effect for professional commitment while occupational anxiety has a negative effect for professional commitment. In addition, having a moderate seniority has a negative (reducing) effect for professional commitment. Other variables (gender, job, and working unit) was found to have no significant impact on professional commitment.

Psychometric properties of the health literacy instrument in Brazil (HLS-EU-BR47).

BACKGROUND: Health literacy (HL) is a key component of health promotion and sustainability and contributes to well-being. Despite its global relevance, HL is an under-researched topic in South America but is now debuting its exploration in Brazil. To leverage its benefits for South America, the mere translation of validated tools into Portuguese is insufficient. Rather, it is necessary to examine their validity. This study aims to assess the psychometric properties of the European Health Literacy Questionnaire (HLS-EU-BR47) using the Item Response Theory (IRT) in a population-based sample of adults in Brazil. METHODS: A cross-sectional online study was conducted across Brazil and included 1028 participants aged 18 years and above (80% women). Cronbach's alpha, McDonald's omega, factor analysis, graded responses model, Item Characteristic Curve, HL levels based on this, HL standard calculation, IRT, and regular score correlation were computed. RESULTS: The instrument exhibit high reliability (Cronbach's alpha 0.95). Factor analysis yielded one factor. IRT was appropriate for data analysis because it allowed quality evaluation of items and constructed a scale to quantify HL. The 47 items and latent features of respondents in the same unit of measurement are positioned in the construction of the HLS-EU-BR47 instrument. The percentages of individuals at each HL level, calculated using IRT, were found to be comparable to those obtained through the standard computation, e.g., 3.2% of people reported very low HL versus 10.8% inadequate HL, 56.2% reported low HL versus 39.5% problematic HL, 31.1% had moderate HL versus 30.1% sufficient HL, and 9.5% had high HL versus 19.7% with excellent HL. The mean HL scores were comparable between women and men (33.9 vs. 33.7, P = 0.36). CONCLUSION: This study provides new evidence of the validity of a widely used HL instrument for the population of South America (in this case, Brazil). This tool can be utilized by citizens, health professionals, and regional/national policymakers to inform the development of initiatives to assess and improve the HL of individuals, groups, and communities. Further studies are needed to confirm and extend the findings and to explore the influence of local cultures and practices in the vast Brazilian territory on HL.

Using a nonparametric item response theory model to identify patterns of cognitive decline: The Mokken scale analysis.

Cognitive decline, particularly in dementia, presents complex challenges in early detection and diagnosis. While Item Response Theory (IRT) has been instrumental in identifying patterns of cognitive impairment through psychometric tests, its parametric models often require large sample sizes and strict assumptions. This creates a need for more adaptable, less demanding analytical methods. This study aimed to evaluate the effectiveness of Mokken scale analysis (MSA), a nonparametric IRT model, in identifying hierarchical patterns of cognitive impairment from psychometric tests. Using data from 1164 adults over 60 years old, we applied MSA to the orientation subscale of ACE-III. Our analysis involved calculating scalability, monotone homogeneity, invariant item ordering (IIO) and response functions. The MSA effectively retrieved the hierarchical order of cognitive impairment patterns. Most items showed strong scalability and consistent patterns of cognitive performance. However, challenges with IIO were observed, particularly with items having adjacent difficulty parameters. The findings highlight MSA's potential as a practical alternative to parametric IRT models in cognitive impairment research. Its ability to provide valuable insights into patterns of cognitive deterioration, coupled with less stringent requirements, makes it a useful tool for clinicians and researchers.

Scaling a common assessment of associative ability: Development and validation of a multiple-choice compound remote associates task.

The assessment of creativity as an individual difference has historically focused on divergent thinking, which is increasingly viewed as involving the associative processes that are also understood to be a key component of creative potential. Research on associative processes has proliferated in many sub-fields, often using Compound Remote Associates (CRA) tasks with an open response format and relatively small participant samples. In the present work, we introduce a new format that is more amenable to large-scale data collection in survey designs, and present evidence for the reliability and validity of CRA measures in general using multiple large samples. Study 1 uses a large, representative dataset (N = 1,323,480) to demonstrate strong unidimensionality and internal consistency (α = .97; ωt = .87), as well as links to individual differences in temperament, cognitive ability, occupation, and job characteristics. Study 2 uses an undergraduate sample (N = 685) to validate the use of a multiple-choice format relative to the traditional approach. Study 3 uses a crowdsourced sample (N = 357) to demonstrate high test-retest reliability of the items (r =.74). Finally, Study 4 uses a sample that overlaps with Study 1 (N = 1,502,922) to provide item response theory (IRT) parameters for a large set of high-quality CRA items that use a multiple-choice response mode, thus facilitating their use in future research on creativity, insight, and related topics.

Development and Validation of the Five-Factor Borderline Inventory-Super Short Form and Screener.

The Five-Factor Borderline Inventory (FFBI) and FFBI-Short Form (FFBI-SF) are 120-item and 48-item measures that assess the underlying maladaptive personality traits of borderline personality disorder (BPD). The purpose of this study was to develop a super short form (FFBI-SSF) and an FFBI-Screener to facilitate the use of dimensional trait measures for BPD. Using item response theory analyses, the 48-item measure was reduced to 22 items using a large undergraduate sample (N = 1300) and then retested using a Mechanical Turk sample (N = 602), demonstrating strong replicability. IRT was again used to further reduce the measure from 22 items to four items to provide a brief screening tool. Correlations of the FFBI-SSF and Screener with measures of BPD-related variables were compared across five samples (N = 919, 204, 580, 281, and 488). Overall, the FFBI-SSF showed similar relations to the FFBI-SF at the full scale and domain-level scales, while the FFBI-screener demonstrated similar relations at the full scale level. This super short form and screener may best be used in large-scale research studies or as part of a screening tool in clinical settings.

The InterModel Vigorish as a Lens for Understanding (and Quantifying) the Value of Item Response Models for Dichotomously Coded Items.

The deployment of statistical models-such as those used in item response theory-necessitates the use of indices that are informative about the degree to which a given model is appropriate for a specific data context. We introduce the InterModel Vigorish (IMV) as an index that can be used to quantify accuracy for models of dichotomous item responses based on the improvement across two sets of predictions (i.e., predictions from two item response models or predictions from a single such model relative to prediction based on the mean). This index has a range of desirable features: It can be used for the comparison of non-nested models and its values are highly portable and generalizable. We use this fact to compare predictive performance across a variety of simulated data contexts and also demonstrate qualitative differences in behavior between the IMV and other common indices (e.g., the AIC and RMSEA). We also illustrate the utility of the IMV in empirical applications with data from 89 dichotomous item response datasets. These empirical applications help illustrate how the IMV can be used in practice and substantiate our claims regarding various aspects of model performance. These findings indicate that the IMV may be a useful indicator in psychometrics, especially as it allows for easy comparison of predictions across a variety of contexts.

The Beck Hopelessness Scale's psychometric features: A new short form based on item response theory.

BACKGROUND: The assessment of hopelessness plays a significant role in preventing various psychological disorders and major life events within the general population. However, the psychometric properties of the Beck Hopelessness Scale (BHS) have been a subject of controversy, primarily studied in clinical groups. The aim of the present study was to gain new psychometric insights and propose a new short version of the BHS for the general population using the Item Response Theory (IRT) approach. METHODS: A total of 2164 Argentinean individuals completed the BHS alongside the Inventory of Suicide Orientation-30. We compared IRT models with two and three parameters for the original BHS version, exploring the removal of redundant and less informative items. Convergent and discriminant validity was also examined. RESULTS: Our results support the 2PL model for the BHS-19. In addition, the BHS-10 short version adequately depicted the same range of the measured trait as the original version, showing reasonable measurement accuracy in the middle-high levels of the trait (marginal reliability = 0.70, Cronbach's α = 0.86). Notably, a positive correlation was found between the factorial score of BHS-10, BHS-19, and suicide orientation. CONCLUSIONS: In conclusion, our findings support the use of a simplified version as a practical and valuable tool for both research and clinical practice in the future.

Ordinal Outcome State-Space Models for Intensive Longitudinal Data.

Intensive longitudinal (IL) data are increasingly prevalent in psychological science, coinciding with technological advancements that make it simple to deploy study designs such as daily diary and ecological momentary assessments. IL data are characterized by a rapid rate of data collection (1+ collections per day), over a period of time, allowing for the capture of the dynamics that underlie psychological and behavioral processes. One powerful framework for analyzing IL data is state-space modeling, where observed variables are considered measurements for underlying states (i.e., latent variables) that change together over time. However, state-space modeling has typically relied on continuous measurements, whereas psychological data often come in the form of ordinal measurements such as Likert scale items. In this manuscript, we develop a general estimation approach for state-space models with ordinal measurements, specifically focusing on a graded response model for Likert scale items. We evaluate the performance of our model and estimator against that of the commonly used "linear approximation" model, which treats ordinal measurements as though they are continuous. We find that our model resulted in unbiased estimates of the state dynamics, while the linear approximation resulted in strongly biased estimates of the state dynamics. Finally, we develop an approximate standard error, termed slice standard errors and show that these approximate standard errors are more liberal than true standard errors (i.e., smaller) at a consistent bias.

A novel scale for assessing the risk of low birthweight: Birthweight questionnaire.

BACKGROUND: The birthweight of a newborn is critical to their health, development, and well-being. Previous studies that used maternal characteristics to predict birthweight did not employ a harmonised scale to assess the risk of low birthweight (LBW). OBJECTIVE: The goal of this study was to develop a new instrument that uses items on a uniform scale to assess the risk of an LBW in a pregnant woman. METHODS: Item response theory was employed to evaluate a similar existing scale, and some weaknesses were identified. RESULTS: Based on the observed weaknesses of the existing scale, a new uniform scale was developed, which is a 3-point Likert scale consisting of seven items. CONCLUSION: The scale, termed birthweight questionnaire, is a valuable tool for collecting data that could assist in assessing the risk of an LBW at every stage of pregnancy.

Development and validation of a research version of the Overall Assessment of the Speaker's Experience of Stuttering- Adult (OASES-A-R).

BACKGROUND: The Overall Assessment of the Speaker's Experience of Stuttering for Adults (OASES-A; Yaruss & Quesal, 2016) is a widely used measurement tool designed to evaluate the adverse impact associated with stuttering. Items examine general perceptions of stuttering, personal reactions to stuttering, functional communication difficulties, and consequences for quality of life. This paper presents a shortened research version of the OASES-A response form (OASES-A-R) that can be used by researchers in scientific studies involving adults who stutter that reflect the Section and Total Scores of the original OASES-A using fewer items. METHOD: Previously collected OASES-A data (N = 315) were analyzed via graded response modeling to identify discrimination values of each OASES-A item in measuring each OASES-A Section Total Score. Items with the highest discrimination and items judged by expert clinicians to be more important in measuring adverse impact (N = 27) were used to create a shortened OASES-A-R. The shortened OASES-A-R response form was then validated and compared to the full OASES-A response form in an independent sample (N = 156). RESULTS: The shortened 25-item OASES-A-R response form demonstrated very high and positive correlations with the full OASES-A response form. Similarly, each OASES-A-R Section demonstrated high internal reliability coefficients similar to those of the OASES-A. DISCUSSION: The resulting 25-item OASES-A-R response form provides a reflection of the speaker's experience of stuttering as measured by the original 100-item OASES-A that is suitable for use in certain research studies of adults who stutter. Clinical use is not recommended, as the full OASES-A provides additional insights about a client's experience of stuttering that are necessary for effective treatment planning and intervention.

[Study on the applicability of the Brief Job Stress Questionnaire based on classical test theory and item response theory].

Objective: To evaluate the quality and item characteristics of the Brief Job Stress Questionnaire (BJSQ) among employees in manufacturing and service industries. Methods: From December 2021 to December 2022, a total of 2077 employees from 4 manufacturing and service enterprises in Beijing were selected by the method of combining purpose sampling and convenience sampling. The Chinese version of BJSQ was used to carry out a survey on occupational stress from 57 items in 4 dimensions including job stress factors, stress response, social support factors and satisfaction. Classical test theory (CTT) was used to analyze the validity and reliability of the questionnaire. The Semejima hierarchical response model in item response theory (IRT) was used to analyze the differentiation a, difficulty coefficient b and information content of each item. Results: Among the 2077 subjects, the age M (Q(1), Q(3)) was 33 (28, 37) years old, and the working age M (Q(1), Q(3)) was 4 (2, 8) years. There were 723 (34.8%) people in manufacturing industry and 1354 (65.2%) people in service industry. Eleven common factors were extracted by exploratory factor analysis, with a cumulative variance contribution rate of 62.823%, the variance of the common factors of each entry ranging from 0.451 to 0.865, and the range of factor loading values from 0.413 to 0.825, with 5 items having cross-loadings. The results of the validation factor analysis showed that the model fit indexes of root means square error of approximation was 0.055, comparative fit index was 0.950, Tucker Lewis index was 0.948, and standardized root mean square was 0.066. Content validity results showed that the total scores of the Chinese version of the BJSQ were positively correlated with the scores of the 4 dimensions (r(s)=0.487-0.936, P<0.05) . The results of the reliability analysis showed that the Cronbach's alpha coefficient for the total questionnaire was 0.945, and the Cronbach's alpha coefficients of job stress factors, stress response and social support factors were 0.775, 0.957, and 0.830, respectively. The Spearman-Brown coefficient for the total questionnaire was 0.866, and for the 3 dimensions, it was 0.572, 0.882 and 0.772. The results of IRT analysis showed that only 1 of the 57 items had a differentiation a-value<0.30, the difficulty coefficient b-value of each item ranged from -12.02 to 11.09, of which 8 items had a difficulty coefficient that was too high, 3 items had a difficulty coefficient that was too low, and 3 items did not meet the requirements, and the average amount of information in each item ranged from 0.022 to 2.566, and there were 47 items with average amount of information>0.088 (5/57) . Conclusion: The Chinese version of BJSQ has good reliability and validity in the typical occupational groups of manufacturing and service industries in China, and most of the items have good performance, but some items still need to be further improved and optimized or deleted.

Psychometric evaluation of the pictorial scale of perceived movement skill competence in Chinese children: An item response theory analysis.

AIM: This study aimed to investigate the psychometric characteristics of the Pictorial Scale of Perceived Movement Skill Competence (PMSC) in Chinese children, utilizing the principles of item response theory (IRT). METHODS: A total of 284 children (147 boys), aged 4-10 years (Mage = 7.39, SD = 2.18), from four schools were recruited for testing, adhering to the principle of random sampling. The Rasch model was applied to evaluate the rating scale validity, local independence, scale dimensionality, person and item fit statistics, and differential item functioning (DIF). RESULTS: (1) The rating scale validity was confirmed by the monotonically increasing trend in both average item difficulty measures and Andrich thresholds. (2) However, the observation that standardized residual correlations for all items remained below 0.3 unequivocally confirmed the presence of local independence among the variables. (3) The scale's dimensions were substantiated, with Rasch measures explaining 40.6% of total variance, exceeding the 40% threshold. Additionally, the first contrast had an eigenvalue of 1.8 and explained 8.80% of the variance, both exceeding the minimum 3:1 ratio. Consequently, the unidimensionality assumption of the item was supported. (4) Person and item fit statistics indicated requisite reliability and separation, with all achieving suitable infit and outfit measures, and point-biserial correlations exceeding 0.55. (5) Regarding gender DIF, it was observed that the majority of items did not exhibit statistically significant gender DIF effects (p > 0.05) except item 8 (Bounce a ball, t = -2.74, p < 0.01, DIF = -0.51). CONCLUSION: Rasch analysis revealed that the instrument exhibited strong scale validity among Chinese children, adhering to the Rasch model's unidimensionality and local independence principles, with robust person and item reliability and separation. All items demonstrated a good fit. Additionally, the significant gender difference in item 8 (Bounce a ball) warrants further consideration. In conclusion, PMSC is suitable for assessing the PMSC of Chinese children.

Psychometric properties of the chinese version of multidimensional experiential avoidance questionnaire-30.

BACKGROUND: Experiential avoidance represents the tendency to avoid negative internal experiences, which is a key concept in Acceptance and Commitment Therapy. However, existing measures of experiential avoidance (i.e., Acceptance and Action Questionnaire-II, AAQ-II) have some limitations. This study aims to assess the psychometric properties of the Chinese version of Multidimensional Experiential Avoidance Questionnaire-30 (MEAQ-30) and provide evidence for the reliability and validity of this new instrument. METHODS: Two questionnaire surveys were conducted. The first sample (N = 546) was analyzed using classical test theory (CTT), and the second sample (N = 511) was analyzed using multidimensional item response theory (MIRT). RESULTS: CTT supported the six-factor structure of MEAQ-30, indicating good internal consistency and measurement invariance across genders. Furthermore, the Chinese version of MEAQ-30 showed satisfactory convergent and discriminant validity. The incremental validity test showed that after controlling for the effects of neuroticism and AAQ-II, the Chinese version of MEAQ-30 could still significantly predict depression, anxiety, and stress. MIRT indicated that 30 items had good discrimination and difficulty, and the six subscales were sufficiently reliable across the continuum of experiential avoidance. CONCLUSION: The Chinese version of MEAQ-30 has good reliability and validity and is suitable for assessing experiential avoidance among Chinese populations.

The Crosswise Model for Surveys on Sensitive Topics: A General Framework for Item Selection and Statistical Analysis.

When surveys contain direct questions about sensitive topics, participants may not provide their true answers. Indirect question techniques incentivize truthful answers by concealing participants' responses in various ways. The Crosswise Model aims to do this by pairing a sensitive target item with a non-sensitive baseline item, and only asking participants to indicate whether their responses to the two items are the same or different. Selection of the baseline item is crucial to guarantee participants' perceived and actual privacy and to enable reliable estimates of the sensitive trait. This research makes the following contributions. First, it describes an integrated methodology to select the baseline item, based on conceptual and statistical considerations. The resulting methodology distinguishes four statistical models. Second, it proposes novel Bayesian estimation methods to implement these models. Third, it shows that the new models introduced here improve efficiency over common applications of the Crosswise Model and may relax the required statistical assumptions. These three contributions facilitate applying the methodology in a variety of settings. An empirical application on attitudes toward LGBT issues shows the potential of the Crosswise Model. An interactive app, Python and MATLAB codes support broader adoption of the model.

An Item Response Theory Model for Incorporating Response Times in Forced-Choice Measures.

Forced-choice (FC) measures have been widely used in many personality or attitude tests as an alternative to rating scales, which employ comparative rather than absolute judgments. Several response biases, such as social desirability, response styles, and acquiescence bias, can be reduced effectively. Another type of data linked with comparative judgments is response time (RT), which contains potential information concerning respondents' decision-making process. It would be challenging but exciting to combine RT into FC measures better to reveal respondents' behaviors or preferences in personality measurement. Given this situation, this study aims to propose a new item response theory (IRT) model that incorporates RT into FC measures to improve personality assessment. Simulation studies show that the proposed model can effectively improve the estimation accuracy of personality traits with the ancillary information contained in RT. Also, an application on a real data set reveals that the proposed model estimates similar but different parameter values compared with the conventional Thurstonian IRT model. The RT information can explain these differences.

Wald χ2 Test for Differential Item Functioning Detection with Polytomous Items in Multilevel Data.

Identifying items with differential item functioning (DIF) in an assessment is a crucial step for achieving equitable measurement. One critical issue that has not been fully addressed with existing studies is how DIF items can be detected when data are multilevel. In the present study, we introduced a Lord's Wald χ2 test-based procedure for detecting both uniform and non-uniform DIF with polytomous items in the presence of the ubiquitous multilevel data structure. The proposed approach is a multilevel extension of a two-stage procedure, which identifies anchor items in its first stage and formally evaluates candidate items in the second stage. We applied the Metropolis-Hastings Robbins-Monro (MH-RM) algorithm to estimate multilevel polytomous item response theory (IRT) models and to obtain accurate covariance matrices. To evaluate the performance of the proposed approach, we conducted a preliminary simulation study that considered various conditions to mimic real-world scenarios. The simulation results indicated that the proposed approach has great power for identifying DIF items and well controls the Type I error rate. Limitations and future research directions were also discussed.