Setting a standard for low reading proficiency: A comparison of the bookmark procedure and constrained mixture Rasch model.

In order to draw pertinent conclusions about persons with low reading skills, it is essential to use validated standard-setting procedures by which they can be assigned to their appropriate level of proficiency. Since there is no standard-setting procedure without weaknesses, external validity studies are essential. Traditionally, studies have assessed validity by comparing different judgement-based standard-setting procedures. Only a few studies have used model-based approaches for validating judgement-based procedures. The present study addressed this shortcoming and compared agreement of the cut score placement between a judgement-based approach (i.e., Bookmark procedure) and a model-based one (i.e., constrained mixture Rasch model). This was performed by differentiating between individuals with low reading proficiency and those with a functional level of reading proficiency in three independent samples of the German National Educational Panel Study that included students from the ninth grade (N = 13,897) as well as adults (Ns = 5,335 and 3,145). The analyses showed quite similar mean cut scores for the two standard-setting procedures in two of the samples, whereas the third sample showed more pronounced differences. Importantly, these findings demonstrate that model-based approaches provide a valid and resource-efficient alternative for external validation, although they can be sensitive to the ability distribution within a sample.

More Than (Single) Text Comprehension? - On University Students' Understanding of Multiple Documents.

The digital revolution has made a multitude of text documents from highly diverse perspectives on almost any topic easily available. Accordingly, the ability to integrate and evaluate information from different sources, known as multiple document comprehension, has become increasingly important. Because multiple document comprehension requires the integration of content and source information across texts, it is assumed to exceed the demands of single text comprehension due to the inclusion of two additional mental representations: the integrated situation model and the intertext model. To date, there is little empirical evidence on commonalities and differences between single text and multiple document comprehension. Although the relationships between single text and multiple document comprehension can be well distinguished conceptually, there is a lack of empirical studies supporting these assumptions. Therefore, we investigated the dimensional structure of single text and multiple document comprehension with similar test setups. We examined commonalities and differences between the two forms of text comprehension in terms of their relations to final school exam grades, level of university studies and university performance. Using a sample of n = 501 students from two German universities, we jointly modeled single text and multiple document comprehension and applied a series of regression models. Concerning the relationship between single text and multiple document comprehension, confirmatory dimensionality analyses revealed the best fit for a model with two separate factors (latent correlation: 0.84) compared to a two-dimensional model with cross-loadings and fixed covariance between the latent factors and a model with a general factor. Accordingly, the results indicate that single text and multiple document comprehension are separable yet correlated constructs. Furthermore, we found that final school exam grades, level of university studies and prior university performance statistically significant predicted both single text and multiple document comprehension and that expected future university performance was predicted by multiple document comprehension. There were also statistically significant relationships between multiple document comprehension and these variables when single text comprehension was taken into account. The results imply that multiple document comprehension is a construct that is closely related to single text comprehension yet empirically differs from it.

Validating process variables of sourcing in an assessment of multiple document comprehension.

BACKGROUND: With digital technologies, competence assessments can provide process data, such as mouse clicks with corresponding timestamps, as additional information about the skills and strategies of test takers. However, in order to use variables generated from process data sensibly for educational purposes, their interpretation needs to be validated with regard to their intended meaning. AIMS: This study seeks to demonstrate how process data from an assessment of multiple document comprehension can be used to represent sourcing, which summarizes activities for the consideration of the origin and intention of documents. The investigated process variables were created according to theoretical assumptions about sourcing, and systematically tested for differences between persons, units (i.e., documents and items), and properties of the test administration. SAMPLE: The sample included 310 German university students (79.4% female), enrolled in several bachelor's or master's programmes of the social sciences and humanities. METHODS: Regarding the hierarchical data structure, the hypotheses were analysed with generalized linear mixed models (GLMM). RESULTS: The results mostly revealed expected differences between individuals and units. However, unexpected effects of the administered order of units and documents were detected. CONCLUSIONS: The study demonstrates the theory-informed construction of process variables from log-files and an approach for empirical validation of their interpretation. The results suggest that students apply sourcing for different reasons, but also stress the need of further validation studies and refinements in the operationalization of the indicators investigated.

New Challenge, New Motivation? Goal Orientation Development in Graduates of Higher Track Schools and Their Peers in Vocational Training.

Many studies have demonstrated a decrease in mastery-approach goals and an increase in performance-approach goals after students' transition from primary to secondary education. A theoretical explanation for this phenomenon is a deteriorating fit between a learner's needs and environmental conditions. The purpose of this study was to further examine the development of students' goal orientation after they graduated from a higher track secondary school and transitioned to university or vocational training as compared with peers who chose vocational training earlier. We also examined the fit between the students' needs and the conditions in the new educational context to elaborate on the differential fit hypothesis. Data from 487 students and trainees who participated in a German longitudinal school study were available for our analyses. Latent change score models indicated a significant increase in mastery-approach and a decrease in performance-approach goals for higher track graduates after they transitioned to a new educational context, paralleled by an adequate fit between the learners' needs and the new educational context. For their peers who started vocational training early, mastery-approach goals seem to remain stable, whereas performance-approach goals decreased over time. The results are discussed in the context of the stage-environment fit theory.

Training Early Literacy Related Skills: To Which Degree Does a Musical Training Contribute to Phonological Awareness Development?

Well-developed phonological awareness skills are a core prerequisite for early literacy development. Although effective phonological awareness training programs exist, children at risk often do not reach similar levels of phonological awareness after the intervention as children with normally developed skills. Based on theoretical considerations and first promising results the present study explores effects of an early musical training in combination with a conventional phonological training in children with weak phonological awareness skills. Using a quasi-experimental pretest-posttest control group design and measurements across a period of 2 years, we tested the effects of two interventions: a consecutive combination of a musical and a phonological training and a phonological training alone. The design made it possible to disentangle effects of the musical training alone as well the effects of its combination with the phonological training. The outcome measures of these groups were compared with the control group with multivariate analyses, controlling for a number of background variables. The sample included N = 424 German-speaking children aged 4-5 years at the beginning of the study. We found a positive relationship between musical abilities and phonological awareness. Yet, whereas the well-established phonological training produced the expected effects, adding a musical training did not contribute significantly to phonological awareness development. Training effects were partly dependent on the initial level of phonological awareness. Possible reasons for the lack of training effects in the musical part of the combination condition as well as practical implications for early literacy education are discussed.

Attentional and emotional mechanisms of pain processing and their related factors: a structural equations approach.

BACKGROUND/OBJECTIVE: It is known that maladaptive attentional and emotional mechanisms of pain processing - as indicated by constructs such as pain hypervigilance, pain-related anxiety and pain catastrophizing - play an important role in the development and maintenance of chronic pain conditions. However, little is known to date about the potential risk factors for these forms of maladaptive processing. The aim of the present study was to shed more light on this issue. A very comprehensive set of predictor variables was examined in healthy pain-free subjects. METHOD: Participants were 92 young and healthy subjects (mean [+/- SD] age 26.99+/-6.90 years; 47 men, 45 women). Maladaptive attentional and emotional mechanisms of pain processing were assessed by self-report measures of pain hypervigilance, pain-related anxiety and pain catastrophizing, as well as by a dot-probe task. The comprehensive set of predictor variables included measures of affective and bodily distress (depression, anxiety and somatization), experimental pain sensitivity, and cortisol reactivity. Directed relationships were estimated by using structural equation modelling. RESULTS: Structural equation modelling revealed a significant path from affective and bodily distress to self-reported maladaptive attentional and emotional pain processing. In contrast, the paths from pain sensitivity and cortisol reactivity did not reach the level of significance. CONCLUSION: These results support the position that anxiety and depression, as well as somatization, contribute to the aberrance of attentional and emotional mechanisms of pain processing. Surprisingly, the assumption of a close relationship between these maladaptive mechanisms of pain processing and pain sensitivity could not be confirmed.