Temporally Dynamic, Cohort-Varying Value-Added Models.

We aim to estimate school value-added dynamically in time. Our principal motivation for doing so is to establish school effectiveness persistence while taking into account the temporal dependence that typically exists in school performance from one year to the next. We propose two methods of incorporating temporal dependence in value-added models. In the first we model the random school effects that are commonly present in value-added models with an auto-regressive process. In the second approach, we incorporate dependence in value-added estimators by modeling the performance of one cohort based on the previous cohort's performance. An identification analysis allows us to make explicit the meaning of the corresponding value-added indicators: based on these meanings, we show that each model is useful for monitoring specific aspects of school persistence. Furthermore, we carefully detail how value-added can be estimated over time. We show through simulations that ignoring temporal dependence when it exists results in diminished efficiency in value-added estimation while incorporating it results in improved estimation (even when temporal dependence is weak). Finally, we illustrate the methodology by considering two cohorts from Chile's national standardized test in mathematics.

Standard-Setting Procedures for Counts Data.

An examinee- and an item-centered procedure are proposed to set cut scores for counts data. Both procedures assume that the counts data are modelled according to the Rasch Poisson counts model (RPCM). The examinee-centered method is based on Longford's (1996) approach and links contrasting-groups judgements to the RPCM ability scale using a random logistic regression model. In the item-centered method, the judges are asked to describe the minimum performance level of the minimally competent student by giving the minimum number of correct responses (or, equivalently, the maximum number of admissible errors). On the basis of these judgements for each subtest, the position of the minimally competent student on the RPCM ability scale is estimated. Both procedures are illustrated with a standard-setting study on mental arithmetic for students at the end of primary education.

The Use of an Identifiability-Based Strategy for the Interpretation of Parameters in the 1PL-G and Rasch Models.

Using the well-known strategy in which parameters are linked to the sampling distribution via an identification analysis, we offer an interpretation of the item parameters in the one-parameter logistic with guessing model (1PL-G) and the nested Rasch model. The interpretations are based on measures of informativeness that are defined in terms of odds of correctly answering the items. It is shown that the interpretation of what is called the difficulty parameter in the random-effects 1PL-G model differs from that of the item parameter in a random-effects Rasch model. It is also shown that the traditional interpretation of the guessing parameter in the 1PL-G model changes, depending on whether fixed-effects or random-effects versions of both models are considered.

Modelos Rasch: ¿cuán (in-)coherentemente son presentados y utilizados?

El modelo Rasch es ampliamente usado para el análisis de datos educacionales. En la práctica, se reportan los estimadores de la dificultad de los ítems y los estimadores de las habilidades de los individuos. Sin embargo, nunca se explicita qué significado tiene el término "dificultad" y cuál el término "habilidad". Los significados de estos términos no dependen de las estimaciones; al contrario, estas últimas han de interpretarse en función del significado de los primeros. En este trabajo se muestra que el significado de las nociones de "dificultad" y "habilidad" dependen de la forma en que el modelo Rasch se específica. En la literatura psicométrica hay dos maneras de especificar dicho modelo: una que sólo se limita a modelar los observables; la otra, que además de los observables, modela no-observables. La primera forma de especificación se debe al mismo Rasch, mientras que la segunda fue desarrollada por Lord.

Rasch models are widely used for the analysis of educational data. In practice, estimates of difficulties of items and abilities of examinees are reported. However, the meaning of the terms "difficulty" and "abilities" are never made explicit. The meaning of these terms does not depend on the estimations; they should be interpreted with respect to the eventual meaning of both item difficulties and individual abilities. This paper shows that the meaning of the terms "difficulty" and "ability" depends on the way in which the Rasch model is specified. In the psychometric literature, Rasch models are specified in two different ways: one specifies the observable only, whereas the other one specifies both observable and unobservable. The first specification is due to Rasch himself, the second one is due to Lord.

A Gibbs Sampler for the (Extended) Marginal Rasch Model.

In their seminal work on characterizing the manifest probabilities of latent trait models, Cressie and Holland give a theoretically important characterization of the marginal Rasch model. Because their representation of the marginal Rasch model does not involve any latent trait, nor any specific distribution of a latent trait, it opens up the possibility for constructing a Markov chain - Monte Carlo method for Bayesian inference for the marginal Rasch model that does not rely on data augmentation. Such an approach would be highly efficient as its computational cost does not depend on the number of respondents, which makes it suitable for large-scale educational measurement. In this paper, such an approach will be developed and its operating characteristics illustrated with simulated data.

Erratum to: On the Unidentifiability of the Fixed-Effects 3PL Model.

Condition 2 of Theorem 2 was incorrect in the published version. The correct condition 2 appears in this erratum. Theorem 2. Suppose that I ≥ 3 for the fixed-effects 3PL model. If ω1 = (α1, ß1, c1) is fixed at(1, 0, 0), then1. The person parameters are identified by the observations.2. The item parameters (α2:J , ß2:J , c2:J ) are not identified by the observations.

On the Unidentifiability of the Fixed-Effects 3PL Model.

The paper offers a general review of the basic concepts of both statistical model and parameter identification, and revisits the conceptual relationships between parameter identification and both parameter interpretability and properties of parameter estimates. All these issues are then exemplified for the 1PL, 2PL, and 1PL-G fixed-effects models. For the 3PL model, however, we provide a theorem proving that the item parameters are not identified, do not have an empirical interpretation and that it is not possible to obtain consistent and unbiased estimates of them.

School system evaluation by value added analysis under endogeneity.

Value added is a common tool in educational research on effectiveness. It is often modeled as a (prediction of a) random effect in a specific hierarchical linear model. This paper shows that this modeling strategy is not valid when endogeneity is present. Endogeneity stems, for instance, from a correlation between the random effect in the hierarchical model and some of its covariates. This paper shows that this phenomenon is far from exceptional and can even be a generic problem when the covariates contain the prior score attainments, a typical situation in value added modeling. Starting from a general, model-free definition of value added, the paper derives an explicit expression of the value added in an endogeneous hierarchical linear Gaussian model. Inference on value added is proposed using an instrumental variable approach. The impact of endogeneity on the value added and the estimated value added is calculated accurately. This is also illustrated on a large data set of individual scores of about 200,000 students in Chile.

Identification of the 1PL model with guessing parameter: parametric and semi-parametric results.

In this paper, we study the identification of a particular case of the 3PL model, namely when the discrimination parameters are all constant and equal to 1. We term this model, 1PL-G model. The identification analysis is performed under three different specifications. The first specification considers the abilities as unknown parameters. It is proved that the item parameters and the abilities are identified if a difficulty parameter and a guessing parameter are fixed at zero. The second specification assumes that the abilities are mutually independent and identically distributed according to a distribution known up to the scale parameter. It is shown that the item parameters and the scale parameter are identified if a guessing parameter is fixed at zero. The third specification corresponds to a semi-parametric 1PL-G model, where the distribution G generating the abilities is a parameter of interest. It is not only shown that, after fixing a difficulty parameter and a guessing parameter at zero, the item parameters are identified, but also that under those restrictions the distribution G is not identified. It is finally shown that, after introducing two identification restrictions, either on the distribution G or on the item parameters, the distribution G and the item parameters are identified provided an infinite quantity of items is available.

The local spatial autocorrelation and the kernel method for identifying black zones. A comparative approach.

This article aims to determine the location and the length of road sections characterized by a concentration of accidents (black zones). Two methods are compared: one based on a local decomposition of a global autocorrelation index, the other on kernel estimation. After explanation, both methods are applied and compared in terms of operational results, respective advantages and shortcomings, as well as underlying conceptual elements. The operationality of both methods is illustrated by an application to one Belgian road.