Incorporating calibration errors in oral reading fluency scoring.

Oral reading fluency (ORF) assessments are commonly used to screen at-risk readers and evaluate interventions' effectiveness as curriculum-based measurements. Similar to the standard practice in item response theory (IRT), calibrated passage parameter estimates are currently used as if they were population values in model-based ORF scoring. However, calibration errors that are unaccounted for may bias ORF score estimates and, in particular, lead to underestimated standard errors (SEs) of ORF scores. Therefore, we consider an approach that incorporates the calibration errors in latent variable scores. We further derive the SEs of ORF scores based on the delta method to incorporate the calibration uncertainty. We conduct a simulation study to evaluate the recovery of point estimates and SEs of latent variable scores and ORF scores in various simulated conditions. Results suggest that ignoring calibration errors leads to underestimated latent variable score SEs and ORF score SEs, especially when the calibration sample is small.

Equating Oral Reading Fluency Scores: A Model-Based Approach.

Words read correctly per minute (WCPM) is the reporting score metric in oral reading fluency (ORF) assessments, which is popularly utilized as part of curriculum-based measurements to screen at-risk readers and to monitor progress of students who receive interventions. Just like other types of assessments with multiple forms, equating would be necessary when WCPM scores are obtained from multiple ORF passages to be compared both between and within students. This article proposes a model-based approach for equating WCPM scores. A simulation study was conducted to evaluate the performance of the model-based equating approach along with some observed-score equating methods with external anchor test design.

Harmonizing Assessments of Everyday Racial Discrimination Experiences: The Multigroup Everyday Racial Discrimination Scale (MERDS).

Reliable and valid assessment of direct racial discrimination experiences in everyday life is critical to understanding one key determinant of ethnoracial minority health and health disparities. To address psychometric limitations of existing instruments and to harmonize the assessment of everyday racial discrimination, the new Multigroup Everyday Racial Discrimination Scale (MERDS) was developed and validated. This investigation included 1,355 college and graduate students of color (Mage = 21.54, 56.0% women). Factor analyses were performed to provide evidence for structural validity of everyday racial discrimination scores. Item response theory modeling was used to investigate item difficulty relative to the level of everyday racial discrimination, and measurement error conditioned on the construct. MERDS scores were reliable, supported construct unidimensionality, and distinguished individuals who reported low to very high frequency of everyday racial discrimination. Results on the associations with racial identity and psychopathology symptoms, and utility of the scale are discussed.

Penalized likelihood methods for modeling count data.

The paper considers parameter estimation in count data models using penalized likelihood methods. The motivating data consists of multiple independent count variables with a moderate sample size per variable. The data were collected during the assessment of oral reading fluency (ORF) in school-aged children. A sample of fourth-grade students were given one of ten available passages to read with these differing in length and difficulty. The observed number of words read incorrectly (WRI) is used to measure ORF. Three models are considered for WRI scores, namely the binomial, the zero-inflated binomial, and the beta-binomial. We aim to efficiently estimate passage difficulty, a quantity expressed as a function of the underlying model parameters. Two types of penalty functions are considered for penalized likelihood with respective goals of shrinking parameter estimates closer to zero or closer to one another. A simulation study evaluates the efficacy of the shrinkage estimates using Mean Square Error (MSE) as metric. Big reductions in MSE relative to unpenalized maximum likelihood are observed. The paper concludes with an analysis of the motivating ORF data.

Acculturation and daily cigarette use among Mexican-origin youth: The moderating role of executive functions.

INTRODUCTION: Acculturation and enculturation have been conceptualized, respectively, as risk and protective factors for cigarette use. Although acculturation/enculturation orientations are often studied as stable characteristics, they represent a dynamic process influenced by individuals' social environments and can fluctuate across time. Therefore, investigating how youth actively navigate their acculturation and enculturation beliefs and behaviors on a day-to-day basis can advance scientific understanding of factors related to cigarette use. Executive functions, including inhibitory control, shifting, and working memory, are robust predictors of smoking (e.g., cigarette use). However, we know little about the protective role of executive functions on the daily level associations between acculturation/enculturation and cigarette use among Mexican-origin youth. OBJECTIVES: In a low-income Mexican-origin youth sample (M = 16.94, SD = 1.01; 52 % female), this study examined within-person associations between daily acculturation/enculturation and daily cigarette use and the moderating role of individual-level executive functions. METHOD: We captured the daily fluctuations of acculturation/enculturation and smoking by utilizing data from a 4-day daily diary. The study assessed inhibitory control, shifting, and working memory using behavioral paradigms. RESULTS: A multilevel logistic moderation model revealed statistically significant interactions between acculturation (but not enculturation) and all executive function skills predicting cigarette use. Higher daily acculturation levels were related to greater odds of daily cigarette use only for youth with lower levels of executive function skills. CONCLUSION: Findings suggest that interventions aimed at improving executive functions may protect Mexican-origin youth from the possible adverse effect of acculturation on cigarette use.

Bias for Treatment Effect by Measurement Error in Pretest in ANCOVA Analysis.

This paper investigated consequences of measurement error in the pretest on the estimate of the treatment effect in a pretest-posttest design with the analysis of covariance (ANCOVA) model, focusing on both the direction and magnitude of its bias. Some prior studies have examined the magnitude of the bias due to measurement error and suggested ways to correct it. However, none of them clarified how the direction of bias is affected by measurement error. This study analytically derived a formula for the asymptotic bias for the treatment effect. The derived formula is a function of the reliability of the pretest, the standardized population group mean difference for the pretest, and the correlation between pretest and posttest true scores. It revealed a concerning consequence of ignoring measurement errors in pretest scores: treatment effects could be overestimated or underestimated, and positive treatment effects can be estimated as negative effects in certain conditions. A simulation study was also conducted to verify the derived bias formula.

Pathways linking ethnic discrimination and drug-using peer affiliation to underage drinking status among Mexican-origin adolescents.

Using a three-wave longitudinal data set of Mexican-origin adolescents (N = 602, Mage = 12.92, SD = 0.91 at Wave 1), this study examines parallel pathways from early exposure to ethnic discrimination and drug-using peers, separately, to underage drinking status by late adolescence. Negative affect was expected to mediate the link from ethnic discrimination to underage drinking status (the stress-induced pathway), whereas social alcohol expectancy was expected to mediate the link from drug-using peers to underage drinking status (the socialization pathway). Our findings lend support to the stress-induced pathway while controlling for the socialization pathway. For the stress-induced pathway, we found that early ethnic discrimination experiences were related to higher likelihood of having engaged in underage drinking by late adolescence through elevated negative affect sustained across adolescence. For the socialization pathway, we found no association between affiliation with drug-using peers in early adolescence and underage drinking status, either directly or indirectly. Present findings highlight the unique role of early ethnic discrimination experiences in underage drinking among Mexican-origin adolescents, over and above the effect of drug-using peers. Alcohol use interventions targeting ethnic minority adolescents should account for adolescents' ethnic discrimination experiences by helping adolescents develop adaptive coping strategies to handle negative affect induced by discrimination (e.g., reappraisal) rather than using alcohol to self-medicate. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Self-Blame in Adolescents Who Have Been Sexually Abused: Factor Structure and Differential Correlates of Abuse-Specific and Global Measures.

Self-blame appraisals are frequently studied among adolescents following sexual abuse. However, the conceptualization and operationalization of self-blame varies across studies, with some examining self-blame specific to the abuse and others examining global self-blame. The present study examined the factor structure and theorized correlates of measures of self-blame appraisals among a sample of adolescents who had been sexually abused (N = 493, 91% female). Results of confirmatory factor analyses indicated that a two-factor model, in which abuse-specific and global self-blame appraisals load onto separate factors, produced a superior model fit compared with a single-factor model, though the two factors were highly correlated. Abuse-specific and global self-blame appraisals are differentially associated with theorized correlates, such as experiencing coercion during the abuse. Taken together, the findings suggest that adolescents' abuse-specific and global self-blame appraisals following sexual abuse are measuring distinct constructs.

Development, item analysis, and initial reliability and validity of three forms of a multiple-choice mental health literacy assessment for college students (MHLA-c).

Despite increasing rates of mental health disorders among college students, there are a limited number of validated mental health literacy measures that can be quickly administered and scored in this population. We developed a 54-item multiple-choice measure, consisting of three forms with 18 items on each form. Our items focus on knowledge of more than 20 mental health disorders including their etiology, risk factors, diagnoses, symptoms, treatment, course, and outcome, as well as the application of this knowledge to real world situations. Data were collected on three independent samples of undergraduate students enrolled at an urban public university system in the northeast United States: pilot (n=292), test refinement (n=1,272), and validation (n=683). Basic demographics for the combined test refinement and validation samples were: age=22 ± 4.9 years; 62.2% female; 71.7% non-White. We report on the development of the Mental Health Literacy Assessment-college (MHLA-c) and provide support for its reliability and validity. We also provide descriptive statistics, stratified by gender, college major, and personal experience with a mental health issue to enable its use in diverse settings. The MHLA-c may be useful in measuring knowledge of mental health disorders and related topics among college students. Moreover, the availability of parallel forms will facilitate its use within educational or interventional studies that employ pre-post testing designs.

Evidence for automated scoring and shorter passages of CBM-R in early elementary school.

Curriculum-based measurement of oral reading fluency (CBM-R) is widely used across the United States as a strong indicator of comprehension and overall reading achievement, but has several limitations including errors in administration and large standard errors of measurement. The purpose of this study is to compare scoring methods and passage lengths of CBM-R in an effort to evaluate potential improvements upon traditional CBM-R limitations. For a sample of 902 students in Grades 2 through 4, who collectively read 13,766 passages, we used mixed-effect models to estimate differences in CBM-R scores and examine the effects of (a) scoring method (comparing a human scoring criterion vs. traditional human or automatic speech recognition [ASR] scoring), and (b) passage length (25, 50, or 85 words, and traditional CBM-R length). We also examined differences in word score (correct/incorrect) agreement rates between human-to-human scoring and human-to-ASR scoring. Our results indicated that ASR can be applied in schools to score CBM-R, and that scores for shorter passages are comparable to traditional passages. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Hierarchical Bayes Approach to Estimate the Treatment Effect for Randomized Controlled Trials.

One important issue in Bayesian estimation is the determination of an effective informative prior. In hierarchical Bayes models, the uncertainty of hyperparameters in a prior can be further modeled via their own priors, namely, hyper priors. This study introduces a framework to construct hyper priors for both the mean and the variance hyperparameters for estimating the treatment effect in a two-group randomized controlled trial. Assuming a random sample of treatment effect sizes is obtained from past studies, the hyper priors can be constructed based on the sampling distributions of the effect size mean and precision. The performance of the hierarchical Bayes approach was compared with the empirical Bayes approach (hyperparameters are fixed values or point estimates) and the ordinary least squares (OLS) method via simulation. The design factors for data generation included the sample treatment effect size, treatment/control group size ratio, and sample size. Each generated data set was analyzed using the hierarchical Bayes approach with three hyper priors, the empirical Bayes approach with twelve priors (including correct and inaccurate priors), and the OLS method. Results indicated that the proposed hierarchical Bayes approach generally outperformed the empirical Bayes approach and the OLS method, especially with small samples. When more sample effect sizes were available, the treatment effect was estimated more accurately regardless of the sample sizes. Practical implications and future research directions are discussed.

Estimating Model-Based Oral Reading Fluency: A Bayesian Approach.

Oral reading fluency (ORF), used by teachers and school districts across the country to screen and progress monitor at-risk readers, has been documented as a good indicator of reading comprehension and overall reading competence. In traditional ORF administration, students are given one minute to read a grade-level passage, after which the assessor calculates the words correct per minute (WCPM) fluency score by subtracting the number of incorrectly read words from the total number of words read aloud. As part of a larger effort to develop an improved ORF assessment system, this study expands on and demonstrates the performance of a new model-based estimate of WCPM based on a recently developed latent-variable psychometric model of speed and accuracy for ORF data. The proposed method was applied to a data set collected from 58 fourth-grade students who read four passages (a total of 260 words). The proposed model-based WCPM scores were also evaluated through a simulation study with respect to sample size and number of passages read.

Examining the Pre-service School Principals' Impromptu Speech Skills with a Many-Facet Rasch Model.

The purpose of this study is to demonstrate an application of the many-facet Rasch model (MFRM) in evaluating the impromptu speech skills of pre-service principals in Taiwan. The findings showed that the topics of speech did not exhibit different difficulty measures. With respect to scoring criteria, time control was the most difficult aspect among the scoring criteria. Regarding gender difference in raters, female raters gave lower scores than male raters, but there was no statistical evidence for gender-related bias. However, raters exhibited statistically significant differences in rater severity. The results of this study demonstrates that the MFRM provides a scientific approach to assessment, which can reveal some useful diagnostic information from the original ordinal rating scores on impromptu speech.

Predicting mental health help seeking orientations among diverse Undergraduates: An ordinal logistic regression analysis✰.

BACKGROUND: Recent years have seen a steady increase in college students reporting mental health issues, though only approximately one-third of these students seek treatment. The present study examines: a) students' perceptions of access to campus provided mental health care; b) student stigma attitudes based on social distance and willingness to disclose mental health issues to campus members who might support help-seeking efforts; and c) the predictive value of five factors (aged older than 22, female gender, completed two or more psychology courses, low stigma, and high perception of access) on help-seeking orientation (HSO). METHODS: We performed an ordinal logistic regression (OLR) on data from a diverse sample of undergraduates (n = 1,272). The OLR statistical model is more appropriate for measurement of Likert style data than commonly employed statistical models, which may oversimplify attitudinal data by assuming equal intervals between response categories. RESULTS: Most students did not know that campus-provided counseling was free or confidential, and almost half did not perceive these services as timely or adequate. Students reported more stigma related to disclosing their own problems than to supporting someone else. All five study predictors retained positive and statistically significant slope associations with a positive HSO. Unexpectedly, we found a statistically significant gender interaction with psychology coursework. LIMITATIONS: Data were obtained through self-report measures. CONCLUSIONS: Results are discussed in relation to the possibility that campus-based mental health interventions may remove roadblocks to healthy help-seeking behaviors, particularly for male students.

Elementary teacher's knowledge of response to intervention implementation: a preliminary factor analysis.

In the USA, many states have adopted response to intervention or multi-tiered systems of supports to provide early intervention. However, there is considerable variability in how states and schools implement RTI. Teachers are responsible for using student data from RTI to inform instructional decisions for students with or at risk for dyslexia, so it is necessary to understand the knowledge they have about the structure of RTI in their individual schools. This study reviews the results of an exploratory factor analysis of a survey aimed at measuring teachers' knowledge about RTI implementation and their understanding of RTI implementation within their school. The 52-item survey was administered online to 139 general and special education teachers. The three final factors from this factor analytic work were (1) Teacher Knowledge about Tier 1 Implementation, (2) Teacher Knowledge about Leadership and School Systems, and (3) Teacher Knowledge about Data-Based Decision Making. Factor determinacy scores demonstrated that the survey had high internal consistency. On average, teachers' survey scores were higher on the first two factors and slightly lower on the third factor. Implications of the findings for teachers of students with learning disabilities, including dyslexia, and directions for future research were discussed.

Using propensity score matching to estimate treatment effects of afterschool programs on third-grade reading outcomes.

Evidence on whether afterschool programs enhance academic achievement is limited and mixed. In this study, a group of community providers delivered interventions to students to improve academic achievement. We used propensity score matching to estimate the impact of these afterschool interventions on third-grade reading achievement. A significant effect of moderate magnitude was found favoring students who participated in afterschool programming. Propensity score matching and other methodological considerations are described to support the importance and value of conducting rigorous evaluations of afterschool programs.

Evaluation of Analysis Approaches for Latent Class Analysis with Auxiliary Linear Growth Model.

This study investigated the performance of three selected approaches to estimating a two-phase mixture model, where the first phase was a two-class latent class analysis model and the second phase was a linear growth model with four time points. The three evaluated methods were (a) one-step approach, (b) three-step approach, and (c) case-weight approach. As a result, some important results were demonstrated. First, the case-weight and three-step approaches demonstrated higher convergence rate than the one-step approach. Second, it was revealed that case-weight and three-step approaches generally did better in correct model selection than the one-step approach. Third, it was revealed that parameters were similarly recovered well by all three approaches for the larger class. However, parameter recovery for the smaller class differed between the three approaches. For example, the case-weight approach produced constantly lower empirical standard errors. However, the estimated standard errors were substantially underestimated by the case-weight and three-step approaches when class separation was low. Also, bias was substantially higher for the case-weight approach than the other two approaches.

A two-step sampling weight approach to growth mixture modeling for emergent and developing skills with distributional changes over time.

Emergent reading skills are crucial to the development of fluency and comprehension, and as such, assessing kindergarten entry skills is critical to inform educational decisions. However, skills that are assessed too early are likely to yield many zero scores, as most students do not yet have the experience or ability to perform the task. Although these floor effects typically lessen across time to show near-normal distributions, growth models cannot accommodate repeated measures with different distributions. The purposes of this paper are to (a) introduce a two-step sampling weight approach to growth mixture modeling that addresses distributions changing over time, and (b) apply the approach to a sample of 1911 kindergarten students universally screened on an emergent reading skill (letter sound fluency) across the year. Results distinguish between students that begin at zero and make meaningful gains and those who begin at zero and do not. We discuss the methodological implications of our approach and the practical implications for growth modeling and early identification.

Differential item functioning due to gender between depression and anxiety items among Chilean adolescents.

BACKGROUND: Although much is known about the higher prevalence of anxiety and depressive disorders among adolescent females, less is known about the differential item endorsement due to gender in items of scales commonly used to measure anxiety and depression. AIMS: We conducted a study to examine if adolescent males and females from Chile differed on how they endorsed the items of the Youth Self Report (YSR) anxious/depressed problem scale. We used data from a cross-sectional sample consisting of 925 participants (mean age = 14, SD 1.3, 49% females) of low to lower-middle socioeconomic status. METHODS: A two-parameter logistic (2PL) IRT DIF model was fit. RESULT: s revealed differential item functioning (DIF) by gender for six of the 13 items, with adolescent females being more likely to endorse a depression item while males were found more likely to endorse anxiety items. CONCLUSIONS: Findings suggest that items found in commonly used measures of anxiety and depression symptoms may not equally capture the true levels of these behavioural problems in adolescent males and females. Given the high levels of mental disorders in Chile and the surrounding countries, further attention should be focused on increasing the number of empirical studies examining potential gender differences in the assessment of mental health problems among Latin American populations to better aid our understanding of the phenomenology and determinants of these problems in the region.

The effect of biofeedback training on affective regulation and simulated car-racing performance: a multiple case study analysis.

The foundation of this study was based on an idiosyncratic concept, which uses probabilistic determinations (Kamata, Tenenbaum, & Hanin, 2002) to verify the utility and effectiveness of a biofeedback intervention by manipulating affective performance states in a race-car simulator. Nine males completed five separate time-trials of a simulated racing task and were then randomly assigned to one of three arousal regulation treatment conditions: (1) optimal, (2) poor, and (3) attention control. Following the biofeedback intervention, participants underwent another series of race trials to determine the effectiveness of the arousal regulation intervention. The results indicated that there were relative similarities in the strength and direction of the perceived and physiological states between the participants; however, the subtle details of the participants' unique performance zones and the probability of achieving each zone were revealed to be unique among the participants. The results also indicated that: (a) the biofeedback manipulation resulted in the expected changes for each participant, and (b) there were some large individual differences among the participants, necessitating the idiosyncratic approach. Limitations and future directions are also addressed.