Loevinger on Unidimensional Tests with Reference to Guttman, Rasch, and Wright.

Loevinger's specifications for a unidimensional test are discussed. The implications are reviewed using commentary from Guttman's and Rasch's specification for specific objectivity. A large population is sampled to evaluate the implications of this approach in light of Wright's early presentation regarding data analysis. The results of this analysis show the sample follows the specifications of Loevinger and those of Rasch for a unidimensional test.

Measurement of patient-reported outcomes. 1: The search for the Holy Grail.

Patient-reported outcome measures (PROMs) are used to collect information directly from patients. They may cover several different types of outcomes ranging from symptoms, functioning, utility, satisfaction, through to quality-of-life (QoL). They generally consist of self-completed questionnaires that can be administered by means of hard copies or in a range of electronic formats. PROMs vary considerably in terms of the constructs they assess, the care with which they are developed, and their scientific quality. However, none of the PROMs available approach the quality of measurement achieved by measures/instruments used in physics. PROs are examples of latent variables. These are not directly observable, but can be inferred from, for example, responses to a questionnaire. The only measure of a latent variable that approaches the quality of measurement achieved by the physical sciences is the Lexile Framework for Reading. This framework is based on a construct theory that grew out of an analysis of several available reading measures. A specification equation was generated that was able to link the construct theory to scores obtained with the Lexile measure. A fundamental requirement of this quality of measurement is that the data collected with the model fit Rasch Measurement Theory (RMT). It is argued that PROM developers should aspire to match this level of measurement sophistication if their instruments are to provide valid insights into the impacts of disease and its treatment.

Theory-based metrological traceability in education: A reading measurement network.

Huge resources are invested in metrology and standards in the natural sciences, engineering, and across a wide range of commercial technologies. Significant positive returns of human, social, environmental, and economic value on these investments have been sustained for decades. Proven methods for calibrating test and survey instruments in linear units are readily available, as are data- and theory-based methods for equating those instruments to a shared unit. Using these methods, metrological traceability is obtained in a variety of commercially available elementary and secondary English and Spanish language reading education programs in the U.S., Canada, Mexico, and Australia. Given established historical patterns, widespread routine reproduction of predicted text-based and instructional effects expressed in a common language and shared frame of reference may lead to significant developments in theory and practice. Opportunities for systematic implementations of teacher-driven lean thinking and continuous quality improvement methods may be of particular interest and value.

Toward a theory relating text complexity, reader ability, and reading comprehension.

Validity of specification equations used by auto-text processors to estimate theoretical text complexity have increased importance because of the Common Core State Standards. Theoretical estimates of text complexity will inform (a) setting standards for college and career readiness, (b) grade-level standards, matching readers to text, and (d) creating a daily diet of stretch and targeted text designed to grow reading ability and content knowledge. The purpose of this research was to investigate the specification equation used in the Lexile Framework for Reading to measure text complexity. The Lexile Reading Analyzer contains a specification equation that uses proxies for the semantic difficulty and syntactic complexity to estimate the theoretical complexity of professionally-edited text. Differences between theoretical and empirical estimates of text complexity were examined for a set of 446 professionally authored, previously published passages. Students in grades 2-12 read these passages using A Learning Oasis, a web-based technology, to ensure that most of the articles read were well-targeted to student ability (+100L). Each article was response illustrated using an auto-generated semantic cloze item type embedded into passages. Observed student performance on this item type was used to derive an empirical estimate of text complexity for each passage. Theoretical estimates of text complexity accounted for approximately 90 percent of the variance in empirical estimates of text complexity. These findings suggest that the specification equation contains powerful predictors of empirical text complexity, speculation remains on what additional variables might account for the 10 percent of unexplained variation.

Comparison is key.

Several concepts from Georg Rasch's last papers are discussed. The key one is comparison because Rasch considered the method of comparison fundamental to science. From the role of comparison stems scientific inference made operational by a properly developed frame of reference producing specific objectivity. The exact specifications Rasch outlined for making comparisons are explicated from quotes, and the role of causality derived from making comparisons is also examined. Understanding causality has implications for what can and cannot be produced via Rasch measurement. His simple examples were instructive, but the implications are far reaching upon first establishing the key role of comparison.

Causal Rasch models.

Rasch's unidimensional models for measurement show how to connect object measures (e.g., reader abilities), measurement mechanisms (e.g., machine-generated cloze reading items), and observational outcomes (e.g., counts correct on reading instruments). Substantive theory shows what interventions or manipulations to the measurement mechanism can be traded off against a change to the object measure to hold the observed outcome constant. A Rasch model integrated with a substantive theory dictates the form and substance of permissible interventions. Rasch analysis, absent construct theory and an associated specification equation, is a black box in which understanding may be more illusory than not. Finally, the quantitative hypothesis can be tested by comparing theory-based trade-off relations with observed trade-off relations. Only quantitative variables (as measured) support such trade-offs. Note that to test the quantitative hypothesis requires more than manipulation of the algebraic equivalencies in the Rasch model or descriptively fitting data to the model. A causal Rasch model involves experimental intervention/manipulation on either reader ability or text complexity or a conjoint intervention on both simultaneously to yield a successful prediction of the resultant observed outcome (count correct). We conjecture that when this type of manipulation is introduced for individual reader text encounters and model predictions are consistent with observations, the quantitative hypothesis is sustained.

Generally objective measurement of human temperature and reading ability: some corollaries.

We argue that a goal of measurement is general objectivity: point estimates of a person's measure (height, temperature, and reader ability) should be independent of the instrument and independent of the sample in which the person happens to find herself. In contrast, Rasch's concept of specific objectivity requires only differences (i.e., comparisons) between person measures to be independent of the instrument. We present a canonical case in which there is no overlap between instruments and persons: each person is measured by a unique instrument. We then show what is required to estimate measures in this degenerate case. The canonical case encourages a simplification and reconceptualization of validity and reliability. Not surprisingly, this reconceptualization looks a lot like the way physicists and chemometricians think about validity and measurement error. We animate this presentation with a technology that blurs the distinction between instruction, assessment, and generally objective measurement of reader ability. We encourage adaptation of this model to health outcomes measurement.

Rasch-derived latent trait measurement of outcomes: insightful use leads to precision case management and evidence-based practices in functional healthcare.

The use of Rasch-derived latent trait measurement of outcomes for persons with chronic disease and disablement evolved from other fields, particularly education. Person-metrics is the measurement of how much chronic disease and disablement affects an individual's daily activities physically, cognitively, and through vocational and social role participation. The ability of the Rasch model to assume that the probability of a given person/item interaction is governed by the difficulty of the item and the ability of the person is invaluable to disability measurement. The difference between raw scores and true measures is illustrated by an example of a patient whose physical difficulty is rated on rising from a wheelchair and walking 100m (known to be more difficult), and then walking an additional 200m. Though number ratings of 0-1-2 are assigned to these tasks, they are not equidistant, and only a true measure shows the actual levels of physical difficulty.

From model to measurement with dichotomous items.

Psychometric models typically represent encounters between persons and dichotomous items as a random variable with two possible outcomes, one of which can be labeled success. For a given item, the stipulation that each person has a probability of success defines a construct on persons. This model specification defines the construct, but measurement is not yet achieved. The path to measurement must involve replication; unlike coin-tossing, this cannot be attained by repeating the encounter between the same person and the same item. Such replication can only be achieved with more items whose features are included in the model specifications. That is, the model must incorporate multiple items. This chapter examines multi-item model specifications that support the goal of measurement. The objective is to select the model that best facilitates the development of reliable measuring instruments. From this perspective, the Rasch model has important features compared to other models.

How accurate are lexile text measures?

The Lexile Framework for Reading models comprehension as the difference between a reader measure and a text measure. Uncertainty in comprehension rates results from unreliability in reader measures and inaccuracy in text readability measures. Whole-text processing eliminates sampling error in text measures. However, Lexile text measures are imperfect due to misspecification of the Lexile theory. The standard deviation component associated with theory misspecification is estimated at 64L for a standard-length passage (approximately 125 words). A consequence is that standard errors for longer texts (2,500 to 150,000 words) are measured on the Lexile scale with uncertainties in the single digits. Uncertainties in expected comprehension rates are largely due to imprecision in reader ability and not inaccuracies in text readabilities.

Assessment issues in the testing of children at school entry.

The authors introduce readers to the research documenting racial and ethnic gaps in school readiness. They describe the key tests, including the Peabody Picture Vocabulary Test (PPVT), the Early Childhood Longitudinal Study (ECLS), and several intelligence tests, and describe how they have been administered to several important national samples of children. Next, the authors review the different estimates of the gaps and discuss how to interpret these differences. In interpreting test results, researchers use the statistical term "standard deviation" to compare scores across the tests. On average, the tests find a gap of about 1 standard deviation. The ECLS-K estimate is the lowest, about half a standard deviation. The PPVT estimate is the highest, sometimes more than 1 standard deviation. When researchers adjust those gaps statistically to take into account different outside factors that might affect children's test scores, such as family income or home environment, the gap narrows but does not disappear. Why such different estimates of the gap? The authors consider explanations such as differences in the samples, racial or ethnic bias in the tests, and whether the tests reflect different aspects of school "readiness," and conclude that none is likely to explain the varying estimates. Another possible explanation is the Spearman Hypothesis-that all tests are imperfect measures of a general ability construct, g; the more highly a given test correlates with g, the larger the gap will be. But the Spearman Hypothesis, too, leaves questions to be investigated. A gap of 1 standard deviation may not seem large, but the authors show clearly how it results in striking disparities in the performance of black and white students and why it should be of serious concern to policymakers.