Psychometric Properties of the General Movement Optimality Score using Rasch Measurement.

AIM: To explore the psychometric properties of the general movements optimality score (GMOS) by examining its dimensionality, rating scale functioning, and item hierarchies using Rasch measurement. METHODS: Secondary data analysis of the GMOS data for video-recording of 383 infants with uni-, multidimensional, and mixed Rasch partial credit models. Videos were scored based on the global General Movement Assessment categories, and on the amplitude, speed, spatial range, proximal and distal rotations, onset and offset, tremulous and cramped components of the upper and lower extremities (21 items), resulting in the GMOS. RESULTS: The GMOS data fits best to a unidimensional mixed Rasch model with three different classes of infants, with all but two items contributing to the infants' separation. Rating scales functioned well for 19 items. Item difficulty hierarchies varied depending on infants' class. No floor effect and no substantive gaps between item difficulty estimates were found. CONCLUSION: The GMOS has strong psychometric properties to distinguish infants with different functional motor performance and provides a quantitative measure of quality of movement. INTERPRETATION: The GMOS can be confidently used to assist with early diagnosis, grade motor performance, and provide a solid base to study individual general movement developmental trajectories.

Measuring the continuous quality improvement orientation of medical education programs.

PURPOSE: There is a growing interest in applying continuous quality improvement (CQI) methodologies and tools to medical education contexts. One such tool, the "Are We Making Progress" questionnaire from the Malcolm Baldrige National Quality Award framework, adequately captures the dimensions critical for performance excellence and allows organizations to assess their performance and identify areas for improvement. Its results have been widely validated in business, education, and health care and might be applicable in medical education contexts. The measurement properties of the questionnaire data were analyzed using Rasch modeling to determine if validity evidence, based on Messick's framework, supports the interpretation of results in medical education contexts. Rasch modeling was performed since the questionnaire uses Likert-type scales whose estimates might not be amenable to parametric statistical analyses. DESIGN/METHODOLOGY/APPROACH: Leaders and teachers at 16 of the 17 Canadian medical schools were invited in 2015-2016 to complete the 40-item questionnaire. Data were analyzed using the ConQuest Rasch calibration program, rating scale model. FINDINGS: 491 faculty members from 11 (69 percent) schools participated. A seven-dimensional, four-point response scale model better fit the data. Overall data fit to model requirements supported the use of person measures with parametric statistics. The structural, content, generalizability, and substantive validity evidence supported the interpretation of results in medical education contexts. ORIGINALITY/VALUE: For the first time, the Baldrige questionnaire results were validated in medical education contexts. Medical education leaders are encouraged to serially use this questionnaire to measure progress on their school's CQI focus.

Evaluating the Psychometric Properties of a Measure of Ethnic Identity Among Black South African Youth.

The authors examined the dimensionality and psychometric properties of the Multigroup Ethnic Identity Measure (MEIM) among Black South African adolescents (ages = 13-14; 52% female) representing several ethnic groups (Ndebele, Pedi, Sotho, Swati, Tsonga, Tswana, Venda, Xhosa, Zulu) and evaluated the measure for differential item functioning primarily among four ethnic groups (Sotho, Tswana, Xhosa, and Zulu). Results indicated that a two-dimensional model best represented the data, reflecting ethnic search/clarity and ethnic affirmations. Subsequently, we evaluated the equivalence of the MEIM among the four South African ethnic groups and a sample of African American adolescents (Mage  = 15.57 years; SD = 1.22; 51% female). Further analyses revealed that configural and metric models were excellent across the four South African ethnic groups and the African American group. However, scalar invariance (i.e., intercept) was not found; the item intercepts were different for the South African ethnic groups and African Americans. Findings are discussed with consideration for conducting research on ethnic identity among youth in South Africa.

Psychometric Assessment of the Job Embeddedness Instrument: A Rasch Perspective.

The aim of this study was to examine the psychometric properties of the job embeddedness instrument (JEI) using a Rasch perspective in a sample of Registered Nurses (RNs). A secondary analysis of data was conducted from a previous study examining the job embeddedness of rural and urban RNs. A Rasch analysis supported the six underlying dimensions: organizational fit, community fit, organizational links, community links, organizational sacrifice, and community sacrifice. The results of this study also demonstrate additional evidence of the validity, reliability, and generalizability of the JEI inferences with a sample of RNs. In total, 38 of 39 items of the original JEI were retained in the model. The psychometric evaluation attained through this multidimensional Rasch analysis provided support for using the JEI to assess the level of job embeddedness for RNs.

Psychometric Properties and Differential Item Functioning of a Web-Based Assessment of Children's Social Perspective-Taking.

Social perspective-taking (SPT), which involves the ability infer others' intentions, is a consequential social cognitive process. The purpose of this study is to evaluate the psychometric properties of a web-based social perspective-taking (SELweb SPT) assessment designed for children in kindergarten through third grade. Data were collected from two separate samples of children. The first sample included 3224 children and the second sample included 4419 children. Data were calibrated using Rasch dichotomous model (Rasch, 1960). Differential item and test functioning were also evaluated across gender and ethnicity groups. Across both samples, we found: evidence of consistent item fit; unidimensional item structure; and adequate item targeting. Poor item targeting at high and low ability levels suggests that more items are needed to distinguish low and high ability respondents. Analyses of DIF found some significant item-level DIF across gender, but no DIF across ethnicity. The analyses of person measure calibrations with and without DIF items evidenced negligible differential test functioning (DTF) across gender and ethnicity groups in both samples.

Preliminary Development of a Brief Cannabis Use Disorder Screening Tool: The Cannabis Use Disorder Identification Test Short-Form.

Introduction: Rates of cannabis use disorder (CUD) among vulnerable populations have increased in recent years, highlighting a need to equip providers with an efficient screening tool. Materials and Methods: A short form of the Cannabis Use Disorder Identification Test-Revised (CUDIT-R) was developed by using item response theory and traditional statistical methods, with data from two community samples of cannabis users representing two countries. Four item selection methods (Rasch regression, test characteristic curve, logistic regression, discriminant function analysis) were employed to identify the optimal three-item shortened version. The diagnostic ability of the short form was evaluated by using receiver operating characteristic curves. Results: Using a cut score of 2, the 3-item CUDIT-Short Form (CUDIT-SF; reliability alpha=0.66, Sample 1; 0.80, Sample 2) identified 78.26% of participants in Sample 1 and 78.31% of participants in Sample 2 who met DSM-5 criteria for CUD, with 98% agreement in Sample 1 and 93% agreement in Sample 2 with the full CUDIT-R on CUD classifications using a cut score of 13. Specificity was 76.70 and 78.00 in Samples 1 and 2, respectively. Conclusions: The CUDIT-SF may be useful in busy clinical settings for a stepwise screening. Further validation of this shortened version with larger samples and in different settings is warranted.

Beliefs about language development: construct validity evidence.

Understanding language development is incomplete without recognizing children's sociocultural environments, including adult beliefs about language development. Yet there is a need for data supporting valid inferences to assess these beliefs. The current study investigated the psychometric properties of data from a survey (MODeL) designed to explore beliefs in the popular culture, and their alignment with more formal theories. Support for the content, substantive, structural, generalizability, and external aspects of construct validity of the data were investigated. Subscales representing Behaviorist, Cognitive, Nativist, and Sociolinguistic models were identified as dimensions of beliefs. More than half of the items showed a high degree of consensus, suggesting culturally-transmitted beliefs. Behaviorist ideas were most popular. Bilingualism and ethnicity were related to Cognitive and Sociolinguistic beliefs. Identifying these beliefs may clarify the nature of child-directed speech, and enable the design of language intervention programs that are congruent with family and cultural expectations.

Using the Mixed Rasch Model to analyze data from the beliefs and attitudes about memory survey.

In this study, we used the Mixed Rasch Model (MRM) to analyze data from the Beliefs and Attitudes About Memory Survey (BAMS; Brown, Garry, Silver, and Loftus, 1997). We used the original 5-point BAMS data to investigate the functioning of the "Neutral" category via threshold analysis under a 2-class MRM solution. The "Neutral" category was identified as not eliciting the model expected responses and observations in the "Neutral" category were subsequently treated as missing data. For the BAMS data without the "Neutral" category, exploratory MRM analyses specifying up to 5 latent classes were conducted to evaluate data-model fit using the consistent Akaike information criterion (CAIC). For each of three BAMS subscales, a two latent class solution was identified as fitting the mixed Rasch rating scale model the best. Results regarding threshold analysis, person parameters, and item fit based on the final models are presented and discussed as well as the implications of this study.

Validity evidence for the Child Occupational Self Assessment.

To ensure that researchers and practitioners make valid interpretations from children's self-reports, evidence must demonstrate that the self-report has appropriate psychometric properties. The Child Occupational Self Assessment (COSA) is a self-report of occupational competence and value for everyday activities designed to involve children in identifying goals and assessing outcomes. Five hundred two children with disabilities, ages 6-17, completed the COSA. We used a Rasch Partial Credit model and parametric and nonparametric statistics to obtain validity evidence. Evidence suggests that the COSA has good content, structural, and substantive validity as given by item and child fit statistics and unidimensionality evaluation. Evidence for external validity was mixed because child fit status and measures varied with some demographic and assessment administration variables. Evidence suggests that most children's responses to the COSA can be validly interpreted as indicators of occupational competence and value for everyday activities.

Rating scale use by children with disabilities on a self-report of everyday activities.

UNLABELLED: Kramer JM, Smith EV Jr, Kielhofner G. Rating scale use by children with disabilities on a self-report of everyday activities. OBJECTIVE: To examine whether children with disabilities interpret a self-report of perceived competence and importance of everyday activities in a consistent manner and use the rating scales as intended. If not, are differences in how children interpret the scale associated with personal or contextual variables? DESIGN: Assessment and rating scale development using the Mixed Rasch Model. SETTING: Rehabilitation clinics, schools, and research sites in the United States and Europe. PARTICIPANTS: Children (N=407) aged 6 to 17 years; all had a diagnosed disability or received occupational therapy services. INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURE: The Child Occupational Self Assessment; 25 items representing everyday activities and two 4-point scales: competence and importance. RESULTS: For each scale, 2 groups of children were identified. Approximately 50% of the children used the rating scales as intended. The remainder used the scales as reversed 2-point scales; these children were younger and more likely to have an intellectual disability. Country and practice setting were also associated with rating scale use. All items but 1 had acceptable fit to the Rasch model, and groups of children differed in the relative competence and importance reported. CONCLUSIONS: Personal and contextual variables are associated with children with disabilities' use of self-report rating scales. Younger children and children with intellectual disabilities use a modified response pattern.

Caffeine expectancy: instrument development in the Rasch measurement framework.

Although caffeine is the most widely consumed psychoactive drug in the world, the mechanisms associated with consumption are not well understood. Nonetheless, outcome expectancies for caffeine use are thought to underlie caffeine's reinforcing properties. To date, however, there is no available, sufficient measure by which to assess caffeine expectancy. Therefore, the current study sought to develop such a measure employing Rasch measurement models. Unlike traditional measurement development techniques, Rasch analyses afford dynamic and interactive control of the analysis process and generate helpful information to guide instrument construction. A 5-stage developmental process is described, ultimately yielding a 37-item Caffeine Expectancy Questionnaire (CEQ) comprised of 4 factors representing "withdrawal symptoms," "positive effects," "acute negative effects," and "mood effects." Initial evaluation of the CEQ yielded sufficient evidence for various aspects of validity. Although additional research with more heterogeneous samples is required to further assess the measure's reliability and validity, the CEQ demonstrates potential with regard to its utility in experimental laboratory research and clinical application.

The study skills self-efficacy scale for use with Chinese students.

Silver, Smith and Greene (2001) examined the dimensionality of responses to the Study Skills Self-Efficacy Scale (SSSES) using exploratory principal factor analysis (PFA) and Rasch measurement techniques based on a sample of social science students from a community college in the United States. They found that responses defined three related dimensions. In the present study, Messick's (1995) conceptualization of validity was used to organize the exploration of the psychometric properties of data from a Chinese version of the SSSES. Evidence related to the content aspect of validity was obtained via item fit evaluation; the substantive aspect of validity was addressed by examining the functioning of the rating scales; the structural aspect of validity was explored with exploratory PFA and Rasch item fit statistics; and support for the generalizability aspect of validity was investigate via differential item functioning and internal consistency reliability estimates for both items and persons. The exploratory PFA and Rasch analysis of responses to the Chinese version of the SSSES were conducted with a sample of 494 Hong Kong high school students. Four factors emerged including Study Routines, Resource Use, Text-Based Critical Thinking, and Self-Modification. The fit of the data to the Rasch rating scale model for each dimension generally supported the unidimensionality of the four constructs. The ordered average measures and thresholds from the four Rasch analyses supported the continued use of the six-point response format. Item and person reliability were found to be adequate. Differential item functioning across gender and language taught in was minimal.

Selecting a measurement model for the analysis of the National Institutes of Health Stroke Scale.

To select the most appropriate model for the analysis of data from the National Institutes of Health Stroke Scale (NIHSS), the graded-response, Rasch partial credit, and generalized partial credit models were used to analyze NIH stroke data of 1,191 acute ischemic stroke patients. Based on Akaike's Information Criterion (AIC) and Bayesian Information Criterion (BIC), the generalized partial credit model has the most generalizable parameters. Items on the NIHSS have different discriminating powers. The generalized partial credit model, which allows varying slopes of item response functions, is the most appropriate model for the analysis of the NIHSS.

Measuring sexual risk for HIV: a Rasch scaling approach.

In this study, we developed an HIV transmission risk scale and examined its psychometric properties using data on sexual behavior obtained from a probability sample of adult men who have sex with men living in Chicago. We used Messick's (Am Psychol 50:741-749, 1995) conceptualization of unified validity theory to organize the psychometric properties of data. Evidence related to scale content was investigated via Rasch item fit statistics, point-measure correlations, and expert evaluation. The substantive aspect of validity was addressed by interpreting the meaningfulness of the item difficulty hierarchy (continuum of risky behaviors) and assessment of person fit. The structural aspect of validity was assessed using Rasch item fit statistics, principal component analysis of standardized residuals, and other residual analyses. The generalizability aspect of validity was investigated via internal consistency reliability estimates for both items and persons, and aspects of external validity were addressed by examining between-group differences with respect to levels of risky behavior. Applications and suggested future studies are discussed.

Relationship of candidate communication and organization skills to oral certification examination scores.

This study investigated the relationship between candidate's proficiency in communication/organization and ability measures derived from judges' oral examination ratings on a medical specialty certification examination. Judges who rated candidates on the oral examination also provided a separate rating for candidates' communication/organization skills. ANOVA was used to examine differences among levels of communication/organizational skills with respect to candidates' ability measures on the oral examination. There was a statistically significant increase in oral examination measures for candidates with higher levels of communication/organization skills. This supports the hypothesis that candidates who were more proficient in organizing and presenting their responses were more likely to have better performance on the oral examination. Given communication and oral examination ratings were provided by the same judges, future research should investigate whether these preliminary findings generalize to situations in which communication ratings are obtained from independent ratings during and outside the oral examination.

A clinical nutrition course to improve pharmacy students' skills and confidence in counseling patients.

OBJECTIVE: To create, implement, and evaluate a PharmD course on primary care nutrition. DESIGN: A 2-credit hour elective course was offered to second- and third-year pharmacy students. It was informed by the Socratic method using a minimum number of formal lecture presentations and featured problem-based learning exercises, case-based scenarios, and scientific literature to fuel informed debate. A single group posttest design with a retrospective pretest was used to assess students' self-efficacy. ASSESSMENT: There was a significant overall improvement in students' self-efficacy in their ability to practice primary care nutrition. CONCLUSION: Completion of a nutrition course improved students' confidence in providing primary care nutrition and empowered them to speak more comfortably about the role of nutrition in the prevention of chronic diseases.

Instrument development tools and activities for measure validation using Rasch models: part II--validation activities.

Accumulation of validity evidence is an important part of the instrument development process. In Part I of a two-part series, we provided an overview of validity concepts and described how instrument development efforts can be conducted to facilitate the development of validity arguments. In this, Part II of the series, we identify how analyses, especially those conducted within a Rasch measurement framework, can be used to provide evidence to support validity arguments that are founded during the instrument development process.

Measurement properties of the National Institutes of Health Stroke Scale for people with right- and left-hemisphere lesions: further analysis of the clomethiazole for acute stroke study-ischemic (class-I) trial.

OBJECTIVE: To assess the psychometric properties of the National Institutes of Health Stroke Scale (NIHSS) in people with either left or right acute hemisphere stroke for the purpose of improving the scale's sensitivity in detecting neurologic impairment. DESIGN: Secondary analysis of data from the Clomethiazole for Acute Stroke Study-Ischemic using the Rasch partial credit model. We evaluated the data's measurement properties using item-total correlations, Rasch item fit statistics, principle component analysis of standardized person and item residuals, differential item functioning, separation reliability, and the separation ratio. SETTING: Original data were collected in academic and community hospitals as part of a clinical trial. PARTICIPANTS: People with acute ischemic stroke who were seen within 12 hours of onset: 380 people with left-hemisphere stroke and 347 with right-hemisphere stroke. INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURE: The NIHSS. RESULTS: Items of the NIHSS function differently in the right- and left-hemisphere lesion groups. We constructed for each group separate linear scales consisting of a subset of items of the NIHSS to improve its measurement properties. CONCLUSIONS: Our findings provide initial support for the use of individual, targeted scales for measurement of impairment after ischemic stroke. Low person separation reliability may be a consequence of the sample, which included only people with large ischemic cortical strokes.

Instrument development tools and activities for measure validation using Rasch models: part I - instrument development tools.

Instrument development is an arduous task that, if undertaken with care and consideration, can lay the foundation for the development of validity arguments relating to the inferences and decisions that are based on test measures. This article, Part I of a two-part series, provides an overview of validity concepts and describes how instrument development efforts can be conducted to facilitate the development of validity arguments. Our discussion focuses on documentation of the purpose of measurement, creation of test specifications, item development, expert review, and planning of pilot studies. Through these instrument development activities and tools, essential information is documented that will feed into the analysis, summary, and reporting of data relevant to validity arguments discussed in Part II of this series.

Evaluating an Interpersonal Model of Depression among adults with Down syndrome.

The Interpersonal Model of Depression (IMD) based on the Theory of Human Relatedness (Hagerty, Lynch-Sauer, Patusky, & Bouwsema, 1993) is evaluated among adults with Down syndrome. One hundred subjects participated, with 32% having elevated depression scores and 40% stating they felt lonely. The relationship between depression, perceived social support, loneliness, and life satisfaction is statistically significant, F(6, 172) = 4.36, p < .001. Loneliness, social isolation, loss of sense of well-being, self-hate, and social withdrawal are important interpersonal manifestations and represent increasing levels of depression. Social and emotional loneliness are two dimensions of loneliness. The IMD provides a framework to assess depression in this population. Research on the efficacy of depression treatment based on the IMD is needed.