Time-Varying Treatments in Observational Studies: Marginal Structural Models of the Effects of Early Grade Retention on Math Achievement.

One of the main objectives of many empirical studies in the social and behavioral sciences is to assess the causal effect of a treatment or intervention on the occurrence of a certain event. The randomized controlled trial is generally considered the gold standard to evaluate such causal effects. However, for ethical or practical reasons, social scientists are often bound to the use of nonexperimental, observational designs. When the treatment and control group are different with regard to variables that are related to the outcome, this may induce the problem of confounding. A variety of statistical techniques, such as regression, matching, and subclassification, is now available and routinely used to adjust for confounding due to measured variables. However, these techniques are not appropriate for dealing with time-varying confounding, which arises in situations where the treatment or intervention can be received at multiple timepoints. In this article, we explain the use of marginal structural models and inverse probability weighting to control for time-varying confounding in observational studies. We illustrate the approach with an empirical example of grade retention effects on mathematics development throughout primary school.

The effects of early grade retention: Effect modification by prior achievement and age.

This study examines the effects of early grade retention and different effects according to prior achievement and age. Within a population of children at risk of early retention, we compared the development throughout primary school in mathematics achievement after kindergarten retention, first-grade retention, and continuous promotion. Analyzing data from a large-scale longitudinal study using covariate balancing propensity score weighting, the findings revealed that early grade repeaters would score higher in mathematics if they were promoted each year instead. However, the effects diminished or even disappeared in the long term. Compared to kindergarten retention, first-grade retention was found to be more harmful for the mathematics development of younger children specifically.

Enjoying mathematics or feeling competent in mathematics? Reciprocal effects on mathematics achievement and perceived math effort expenditure.

BACKGROUND: The multidimensionality of the academic self-concept in terms of domain specificity has been well established in previous studies, whereas its multidimensionality in terms of motivational functions (the so-called affect-competence separation) needs further examination. AIM: This study aims at exploring differential effects of enjoyment and competence beliefs on two external validity criteria in the field of mathematics. SAMPLE: Data analysed in this study were part of a large-scale longitudinal research project. Following a five-wave design, math enjoyment, math competence beliefs, math achievement, and perceived math effort expenditure measures were repeatedly collected from a cohort of 4,724 pupils in Grades 3-7. METHOD: Confirmatory factor analysis (CFA) was used to test the internal factor structure of the math self-concept. Additionally, a series of nested models was tested using structural equation modelling to examine longitudinal reciprocal interrelations between math competence beliefs and math enjoyment on the one hand and math achievement and perceived math effort expenditure on the other. RESULTS: Our results showed that CFA models with separate factors for math enjoyment and math competence beliefs fit the data substantially better than models without it. Furthermore, differential relationships between both constructs and the two educational outcomes were observed. Math competence beliefs had positive effects on math achievement and negative effects on perceived math effort expenditure. Math enjoyment had (mild) positive effects on subsequent perceived effort expenditure and math competence beliefs. CONCLUSION: This study provides further support for the affect-competence separation. Theoretical issues regarding adequate conceptualization and practical consequences for practitioners are discussed.

First-grade retention in the Flemish educational context: Effects on children's academic growth, psychosocial growth, and school career throughout primary education.

This study examined the effects of first-grade retention on children's academic growth, psychosocial growth, and future school career by following a cohort of first graders until the start of secondary school. The study took place in the Flemish educational context where primary school students are taught in uniform curricular year groups; the same curricular goals are set for all students, irrespective of ability; and grade retention is used as the main way to cater for students not reaching these goals. Propensity score stratification was used to deal with selection bias. Three-level curvilinear growth curve models, encompassing both grade and age comparisons, were used to model children's growth in math skills, reading fluency skills, and psychosocial skills. Two-level logistic regression models were used to model children's likelihood of repeating any grade between Grades 2 and 6, transitioning to a special education primary school, moving to another primary school, and transitioning to the A (versus B) track in secondary education. Overall, results showed that first-grade retention was less helpful for struggling students than generally thought by parents and educators. Limitations of the study and further research suggestions are provided, and practical implications are discussed.

The role of teachers' expectations in the association between children's SES and performance in kindergarten: a moderated mediation analysis.

This study examines the role of teachers' expectations in the association between children's socio-economic background and achievement outcomes. Furthermore, the role of children's ethnicity in moderating this mediated relation is investigated. In the present study, 3,948 children from kindergarten are examined. Data are analysed by means of structural equation modeling. First, results show that teachers' expectations mediate the relation between children's SES and their later language and math achievement, after controlling for children's ethnicity, prior achievement and gender. This result indicates that teachers may exacerbate individual differences between children. Second, children's ethnicity moderates the mediation effect of teachers' expectations with respect to math outcomes. The role of teachers' expectations in mediating the relation between SES and math outcomes is stronger for majority children than for minority children.

Causal ordering of academic self-concept and achievement: effects of type of achievement measure.

BACKGROUND: The relation between academic self-concept and achievement has been examined in a large number of studies. The majority of these studies have found evidence for a reciprocal effects model. However, there is an ongoing debate on how students' achievement should be measured and whether the type of achievement indicator (grades, tests, teacher ratings) affects the causal pattern found in these studies. AIMS: The study aims at clarifying how the types of achievement measures and the way they are modelled can affect the results of causal ordering studies. In that sense, the study will yield recommendations for researchers in this domain and also provide some direction for practitioners seeking ways to enhance their students' achievement and/or academic self-concept. SAMPLE: Repeated measures of academic self-concept and achievement (standardized tests and teacher ratings) were examined in a sample of 1,753 students in Grades 7, 8, 10, and 12. METHOD: Structural equation modelling was used. Several models (with different types and numbers of achievement measures) were compared. RESULTS: Only small differences were found between models using one or two indicators of achievement. All models generally supported the reciprocal effects model. However, the final model, wherein tests and teacher ratings were used as separate latent variables, showed different developmental patterns in the causal relation between academic self-concept and achievement. CONCLUSIONS: Researchers should interpret the results of causal ordering studies discerningly because the type of measure chosen as an indicator of achievement might affect the causal pattern between academic self-concept and achievement.

Classroom problem behavior and teacher-child relationships in kindergarten: the moderating role of classroom climate.

Young children with problem behavior in the classroom are at risk for developing more conflictual and less close relationships with their teachers. Two studies in kindergarten (N=3798; N=237) shed light on some aspects of classroom climate that can moderate this risk for relational problems. Results showed problematic classroom compositions, in terms of high average levels of internalizing or externalizing behavior, to exacerbate the risk for teachers to form more conflictual relationships with children showing externalizing behavior. Additionally, observed emotional support of teachers was found to be protective for the relational functioning of children at risk due to maladjusted behavior. Specifically, with emotionally supportive teachers, children who expose internalizing or externalizing behavior are no longer at risk for developing less close or more conflictual relationships with their teachers respectively. Practical implications and limitations of the studies are reported and suggestions are made for future research.

The Consequence of Ignoring a Level of Nesting in Multilevel Analysis: A Comment.

This short contribution is a comment on M. Moerbeek's exploration of consequences of ignoring a level of clustering in a multilevel model, which was published in the first issue of the 2004 volume of Multivariate Behavioral Research. After having recapitulated the framework and extended the results of Moerbeek's study, we formulate two critical notes. First, we point at the incompleteness of the conclusions drawn by Moerbeek from the analytical work. The second note is concerned with the limitations of the framework itself.