Implementing a Standardized Effect Size in the POLYSIBTEST Procedure.

A study was conducted to implement the use of a standardized effect size and corresponding classification guidelines for polytomous data with the POLYSIBTEST procedure and compare those guidelines with prior recommendations. Two simulation studies were included. The first identifies new unstandardized test heuristics for classifying moderate and large differential item functioning (DIF) for polytomous response data with three to seven response options. These are provided for researchers studying polytomous data using POLYSIBTEST software that has been published previously. The second simulation study provides one pair of standardized effect size heuristics that can be employed with items having any number of response options and compares true-positive and false-positive rates for the standardized effect size proposed by Weese with one proposed by Zwick et al. and two unstandardized classification procedures (Gierl; Golia). All four procedures retained false-positive rates generally below the level of significance at both moderate and large DIF levels. However, Weese's standardized effect size was not affected by sample size and provided slightly higher true-positive rates than the Zwick et al. and Golia's recommendations, while flagging substantially fewer items that might be characterized as having negligible DIF when compared with Gierl's suggested criterion. The proposed effect size allows for easier use and interpretation by practitioners as it can be applied to items with any number of response options and is interpreted as a difference in standard deviation units.

Application of a Longitudinal IRTree Model: Response Style Changes Over Time.

Traditional psychometric models focus on studying observed categorical item responses, but these models often oversimplify the respondent cognitive response process, assuming responses are driven by a single substantive trait. A further weakness is that analysis of ordinal responses has been primarily limited to a single substantive trait at one time point. This study applies a significant expansion of this modeling framework to account for complex response processes across multiple waves of data collection using the item response tree (IRTree) framework. This study applies a novel model, the longitudinal IRTree, for response processes in longitudinal studies, and investigates whether the response style changes are proportional to changes in the substantive trait of interest. To do so, we present an empirical example using a six-item sexual knowledge scale from the National Longitudinal Study of Adolescent to Adult Health across two waves of data collection. Results show an increase in sexual knowledge from the first wave to the second wave and a decrease in midpoint and extreme response styles. Model validation revealed failure to account for response style can bias estimation of substantive trait growth. The longitudinal IRTree model captures midpoint and extreme response style, as well as the trait of interest, at both waves.

Measuring Response Style Stability Across Constructs With Item Response Trees.

Individual response style behaviors, unrelated to the latent trait of interest, may influence responses to ordinal survey items. Response style can introduce bias in the total score with respect to the trait of interest, threatening valid interpretation of scores. Despite claims of response style stability across scales, there has been little research into stability across multiple scales from the beneficial perspective of item response trees. This study examines an extension of the IRTree methodology to include mixed item formats, providing an empirical example of responses to three scales measuring perceptions of social media, climate change, and medical marijuana use. Results show extreme and midpoint response styles were not stable across scales within a single administration and 5-point Likert-type items elicited higher levels of extreme response style than the 4-point items. Latent trait of interest estimation varied, particularly at the lower end of the score distribution, across response style models, demonstrating as appropriate response style model is important for adequate trait estimation using Bayesian Markov chain Monte Carlo estimation.

Reevaluating the SIBTEST Classification Heuristics for Dichotomous Differential Item Functioning.

A simulation study was conducted to investigate the heuristics of the SIBTEST procedure and how it compares with ETS classification guidelines used with the Mantel-Haenszel procedure. Prior heuristics have been used for nearly 25 years, but they are based on a simulation study that was restricted due to computer limitations and that modeled item parameters from estimates of ACT and ASVAB tests from 1987 and 1984, respectively. Further, suggested heuristics for data fitting a two-parameter logistic model (2PL) have essentially went unused since their original presentation. This simulation study incorporates a wide range of data conditions to recommend heuristics for both 2PL and three-parameter logistic (3PL) data that correspond with ETS's Mantel-Haenszel heuristics. Levels of agreement between the new SIBTEST heuristics and Mantel-Haenszel heuristics were similar for 2PL data and higher than prior SIBTEST heuristics for 3PL data. The new recommendations provide higher true-positive rates for 2PL data. Conversely, they displayed decreased true-positive rates for 3PL data. False-positive rates, overall, remained below the level of significance for the new heuristics. Unequal group sizes resulted in slightly larger false-positive rates than balanced designs for both prior and new SIBTEST heuristics, with rates less than alpha levels for equal ability distributions and unbalanced designs versus false-positive rates slightly higher than alpha with unequal ability distributions and unbalanced designs.

Modeling Changes in Response Style with Longitudinal IRTree Models.

Traditional psychometric modeling focuses on observed categorical item responses, which can over-simplify the respondent cognitive response process. A further weakness is that analysis of ordinal responses has been primarily limited to a single substantive trait at one time point. We propose a significant expansion of this modeling framework to account for complex response processes across multiple waves of data collection using the beneficial item response tree framework. This study proposes a novel model, the longitudinal IRTree, for response processes in longitudinal studies, and investigates whether the response style changes are proportional to changes in the substantive trait of interest. A simulation study demonstrates adequate item parameter recovery in a Bayesian framework, especially with larger sample sizes of 2000. The longitudinal change parameters were recovered similarly well, with improved recovery using informative priors over default priors in Mplus. The empirical application demonstrates that relatively stable observed scores are due to a decrease in response styles offsetting an increase in the latent trait of interest.

Explaining Variability in Response Style Traits: A Covariate-Adjusted IRTree.

Contamination of responses due to extreme and midpoint response style can confound the interpretation of scores, threatening the validity of inferences made from survey responses. This study incorporated person-level covariates in the multidimensional item response tree model to explain heterogeneity in response style. We include an empirical example and two simulation studies to support the use and interpretation of the model: parameter recovery using Markov chain Monte Carlo (MCMC) estimation and performance of the model under conditions with and without response styles present. Item intercepts mean bias and root mean square error were small at all sample sizes. Item discrimination mean bias and root mean square error were also small but tended to be smaller when covariates were unrelated to, or had a weak relationship with, the latent traits. Item and regression parameters are estimated with sufficient accuracy when sample sizes are greater than approximately 1,000 and MCMC estimation with the Gibbs sampler is used. The empirical example uses the National Longitudinal Study of Adolescent to Adult Health's sexual knowledge scale. Meaningful predictors associated with high levels of extreme response latent trait included being non-White, being male, and having high levels of parental support and relationships. Meaningful predictors associated with high levels of the midpoint response latent trait included having low levels of parental support and relationships. Item-level covariates indicate the response style pseudo-items were less easy to endorse for self-oriented items, whereas the trait of interest pseudo-items were easier to endorse for self-oriented items.

Psychometric Validation of the 10-Item USDA Food Security Scale for Use with College Students.

Food insecurity is defined as inadequate access to food due to limited resources. Studies regarding college student food insecurity have shown consistently higher rates than the rest of the nation. Many of these studies measure food insecurity using the United States Department of Agriculture's Adult Food Security Survey Module. Despite its prevalence, the module has not been evaluated for use with the college student population. This study uses Rasch analysis, which underlies the current food insecurity classification approach used by the Department of Agriculture, to investigate the Adult Food Security Survey Module's psychometric properties. A sample of 511 students from a public university in the South was used. Findings indicate that the requirements of the Rasch model do not hold for the module with college students. Specifically, the requirements of equal item discrimination and unidimensionality were violated, along with the presence of moderate to large differential item functioning.

Ethical Reasoning Debriefing in Disaster Simulations.

BACKGROUND: With disasters occurring often, nurses must understand and ethically implement disaster management and patient care coordination. Yet these topics are often not discussed in nursing education curricula. Simulations are a potential solution to this ethical educational deficit, allowing students to act as professional nurses in a realistic scenario with minimal threat of harm to themselves or others. AIM: This study investigates the effect of a high fidelity, multiple-casualty disaster simulation followed by a structured faculty-led debriefing session on perceived ethical reasoning confidence on senior Bachelor of Science in Nursing (BSN) students. Additionally, the effect of the intervention on students' perceived importance of ethical reasoning and perceptions of such skills was explored. METHODS: Students were provided with preparatory materials on the START (Simple Triage and Rapid Treatment) System and The Madison Collaborative's Ethical Reasoning in Action Eight Key Questions (8KQ) frameworks one week before the simulation exercise. In total, 90 students worked in pairs during the 15-minute disaster simulation. Participants' ethical reasoning attitudes were measured before and after the exercise, employing the Survey of Ethical Reasoning (SER) to indicate the importance of each of the 8KQ in students' ethical reasoning process using a five-point Likert scale. The SER was administered electronically using Qualtrics and statistical analysis was completed using SPSS. The 8KQ was also used in the debriefing led by faculty. RESULTS: Comparative assessment of pre and post-results demonstrate significant growth in students' ethical reasoning confidence scores (t(89) = -6.609, p < 0.001). CONCLUSIONS: Simulations are shown to be effective educational approaches in developing ethical reasoning confidence and promoting the development of students' ethical preparedness.

Student ethical reasoning confidence pre/post an innovative makerspace course: A survey of ethical reasoning.

BACKGROUND: Challenges today are complex and rapid innovations are required. We instruct a transdisciplinary undergraduate course where engineering, nursing, and pre-professional health students produce tangible innovative solutions to community health challenges using MakerSpace technologies. Students receive evidence-based ethics instruction as part of the course using the 8 Key Questions for improving ethical reasoning. Design thinking, an empathy-based problem solving technique, was used to teach problem solving and provided context for instructing ethical reasoning. OBJECTIVE: The Objective of this research was to assess student ethical reasoning pre/post this course where students concurrently produce innovative products. DESIGN/PARTICIPANTS: Undergraduate students were assessed pre/post course for their perceptions of 1) the importance of, and 2) their confidence in their ability to ethically reason using a digital version of the Survey of Ethical Reasoning, an instrument previously tested in this population. RESULTS: Participants demonstrated a significant gain in their ethical reasoning confidence and maintained their high ranking of the importance of ethical reasoning concurrently to producing innovative products. CONCLUSIONS: It is possible, with deliberate instruction, for transdisciplinary undergraduate students to develop ethical reasoning confidence concurrently to developing innovative products.

Using SAS PROC MCMC for Item Response Theory Models.

Interest in using Bayesian methods for estimating item response theory models has grown at a remarkable rate in recent years. This attentiveness to Bayesian estimation has also inspired a growth in available software such as WinBUGS, R packages, BMIRT, MPLUS, and SAS PROC MCMC. This article intends to provide an accessible overview of Bayesian methods in the context of item response theory to serve as a useful guide for practitioners in estimating and interpreting item response theory (IRT) models. Included is a description of the estimation procedure used by SAS PROC MCMC. Syntax is provided for estimation of both dichotomous and polytomous IRT models, as well as a discussion on how to extend the syntax to accommodate more complex IRT models.

Accuracy and Precision of an Effect Size and Its Variance From a Multilevel Model for Cluster Randomized Trials: A Simulation Study.

This article investigates an effect size (MLM ES) and its variance for cluster randomized trials based on parameter estimates from multilevel modeling analysis. Accuracy and precision of MLM ES were evaluated using Monte Carlo simulation methods and compared with the performance of an effect size, computed from summary statistics, proposed by Hedges (2007; Hedges' dB ). Simulation results indicated that MLM ES had acceptable accuracy in all conditions, also demonstrating efficiency and consistency. With small sample sizes, MLM ES did not suffer from the same negative bias as Hedges' dB due to overestimation of between-cluster variance. With large sample sizes, MLM ES and Hedges' dB were comparable for accuracy and efficiency. Both MLM ES and Hedges' dB showed considerable bias in some conditions when cluster sizes were unequal. An illustrative example using real data was provided.

Multilevel modeling in psychosomatic medicine research.

The primary purpose of this study is to provide an overview of multilevel modeling for Psychosomatic Medicine readers and contributors. The article begins with a general introduction to multilevel modeling. Multilevel regression modeling at two levels is emphasized because of its prevalence in psychosomatic medicine research. Simulated data sets based on some core ideas from the Familias Unidas effectiveness study are used to illustrate key concepts including communication of model specification, parameter interpretation, sample size and power, and missing data. Input and key output files from Mplus and SAS are provided. A cluster randomized trial with repeated measures (i.e., three-level regression model) is then briefly presented with simulated data based on some core ideas from a cognitive-behavioral stress management intervention in prostate cancer.