Development of a Self-Report Measure of Prediction in Daily Life: The Prediction-Related Experiences Questionnaire.

PURPOSE: Predictions are complex, multisensory, and dynamic processes involving real-time adjustments based on environmental inputs. Disruptions to prediction abilities have been proposed to underlie characteristics associated with autism. While there is substantial empirical literature related to prediction, the field lacks a self-assessment measure of prediction skills related to daily tasks. Such a measure would be useful to better understand the nature of day-to-day prediction-related activities and characterize these abilities in individuals who struggle with prediction. METHODS: An interdisciplinary mixed-methods approach was utilized to develop and validate a self-report questionnaire of prediction skills for adults, the Prediction-Related Experiences Questionnaire (PRE-Q). Two rounds of online field testing were completed in samples of autistic and neurotypical (NT) adults. Qualitative feedback from a subset of these participants regarding question content and quality was integrated and Rasch modeling of the item responses was applied. RESULTS: The final PRE-Q includes 19 items across 3 domains (Sensory, Motor, Social), with evidence supporting the validity of the measure's 4-point response categories, internal structure, and relationship to other outcome measures associated with prediction. Consistent with models of prediction challenges in autism, autistic participants indicated more prediction-related difficulties than the NT group. CONCLUSIONS: This study provides evidence for the validity of a novel self-report questionnaire designed to measure the day-to-day prediction skills of autistic and non-autistic adults. Future research should focus on characterizing the relationship between the PRE-Q and lab-based measures of prediction, and understanding how the PRE-Q may be used to identify potential areas for clinical supports for individuals with prediction-related challenges.

Developing and Validating the Preschool Nutrition Education Practices Survey.

OBJECTIVE: Validate the Preschool Nutrition Education Practices Survey. DESIGN: Iterative approach combining design-based research and Standards for Educational and Psychological Testing. SETTING: Los Angeles, CA and Philadelphia, PA Early Care and Education (ECE) classrooms. PARTICIPANTS: Expert panel members (n = 7); ECE teachers: interviews (n = 8), pilot survey (n = 31), and final survey (n = 136). VARIABLES MEASURED: Early care and education nutrition education practices used in the classroom either during class time or mealtime. ANALYSIS: Qualitative content analysis was implemented for content, response process, and consequences of testing validity evidence. Rasch rating scale analysis was conducted for the response process and internal structure validity and reliability evidence. RESULTS: Qualitative field-testing produced strong content, response process, and consequences of testing validity evidence to inform survey modifications. Quantitative field-testing generated a psychometrically sound, well-targeted 12-item survey on a 4-point frequency scale with excellent item and person reliability (0.97 and 0.93 respectively) and separation (5.36 and 3.77 respectively); good Rasch Principal Components Analysis findings (60.3%); and productive item fit statistics (0.50-1.50 logits). CONCLUSIONS AND IMPLICATIONS: Robust validity (content, response process, consequences of testing, internal structure) and reliability evidence were demonstrated for using the Preschool Nutrition Education Practices Survey to assess ECE teachers' use of nutrition education practices. Future research is needed to examine its relationship to other variables, such as nutrition teaching efficacy, and to determine its ability to detect change in ECE nutrition education practices over time and across groups.

A Validity Study Applying the Rasch Model to the American Association for the Advancement of Science Force and Motion Sub-Topic Assessment for Middle School Students.

The purpose of this research was to investigate the reliability of the scores produced and validity of the inferences drawn from the American Association for the Advancement of Science (AAAS, 2018) force and motion sub-topic assessment for middle school students. The assessment of student outcomes in STEM is an international focus in K-12 education. Project 2061, initiated by the AAAS, focuses on addressing challenges related to standards and assessments. This study informs this effort through testing a 14-item multiple-choice test constructed of questions from the AAAS item bank. Two samples of eighth-grade students participated (N = 1777). Rasch analysis applying the dichotomous model (Rasch, 1960) indicated sufficient item separation and reliability. Thirteen items fit the Rasch model and one item was removed for misfit. Further support for construct validity was observed with 78 percent of item ordering aligned with that predicted by physics educators and stability of measures for 11 items across the two samples. One item exhibited significant differential item functioning by gender and minority status in science. After inspection by physics educators, no bias in item wording or context was determined. Recommendation for additional items is made to increase item targeting and variance explained by the Rasch linear measure.

Rasch Derived Teachers' Emotions Questionnaire.

The purpose of this research was to estimate the reliability of the scores produced from and validity of the inferences drawn from the revised 90-item Teachers' Emotion Questionnaire consisting of three measures: frequency of emotional expressivity, self-efficacy for regulation of emotional expressivity when teaching, and self-efficacy for regulation of context-specific emotional expressivity. A void exists in an instrument assessing teachers' regulation and communication of their emotions. One-hundred seventeen practicing teachers participated in this study at Time 1 and 46 at Time 2. Rasch rating scale analyses indicated sufficient item and person separation and reliability and some support for the construct validity of the inferences drawn from the measures. Test re-test reliability for the person estimates was supported for all three measures over a four-week period: r=.592, p<.001, r=.473, p<.01, and r=.641, p<.001, respectively. Concurrent validity for the self-efficacy for regulation of emotional expressivity when teaching measure with the re-appraisal and suppression sub-scales on the Emotional Regulation Questionnaire (Gross and John, 2003) was supported at Time 1. Modifications to rating scales and future directions for assessing teachers' emotions based on these results are discussed.

An experimental study using Rasch analysis to compare absolute magnitude estimation and categorical rating scaling as applied in survey research.

Limited research has applied a measurement model to compare the rating scale functioning of categorical rating scaling (CRS) and absolute magnitude estimation scaling (MES) when rating subjective stimuli. We used an experimental design and applied the Rasch model to the survey data, with each respondent rating items using MES and one of four commonly used agreement-disagreement rating scales. The results indicated that the CRS and MES data were comparable in person and item separation and reliability when the respondents' scales were known. MES had lower standard error for people and items; however MES had disordered step calibrations. Finally, the respondents reported preference of CRS to MES.