A tutorial on automatic post-stratification and weighting in conventional and regression-based norming of psychometric tests.

Norm scores are an essential source of information in individual diagnostics. Given the scope of the decisions this information may entail, establishing high-quality, representative norms is of tremendous importance in test construction. Representativeness is difficult to establish, though, especially with limited resources and when multiple stratification variables and their joint probabilities come into play. Sample stratification requires knowing which stratum an individual belongs to prior to data collection, but the required variables for the individual's classification, such as socio-economic status or demographic characteristics, are often collected within the survey or test data. Therefore, post-stratification techniques, like iterative proportional fitting (= raking), aim at simulating representativeness of normative samples and can thus enhance the overall quality of the norm scores. This tutorial describes the application of raking to normative samples, the calculation of weights, the application of these weights in percentile estimation, and the retrieval of continuous, regression-based norm models with the cNORM package on the R platform. We demonstrate this procedure using a large, non-representative dataset of vocabulary development in childhood and adolescence (N = 4542), using sex and ethnical background as stratification variables.

When biology takes over: TV formats like The Bachelor and The Bachelorette confirm evolutionary theories of partner selection.

Introduction: In this study, we investigated the impact of age on mate selection preferences in males and females, and explored how the formation and duration of committed relationships depend on the sex of the person making the selection. Methods: To this end, we utilized data from the television dating shows The Bachelor and The Bachelorette. In these programs, either a single man ("bachelor") or a woman ("bachelorette") has the opportunity to select a potential long-term partner from a pool of candidates. Our analysis encompassed a total of n = 169 seasons from 23 different countries, beginning with the first airing in 2002. Results: We found that the likelihood of the final couple continuing their relationship beyond the broadcast was higher in The Bachelorette than in The Bachelor, although the duration of these relationships was not significantly influenced by the type of show. On average, women were younger, both when selecting their partner and when being chosen. However, men exhibited a greater preference for larger age differences than women. Furthermore, the age of the chosen male partners significantly increased with the age of the "bachelorettes," whereas "bachelors" consistently favored women around 25.5 years old, regardless of their own age. Discussion: We discuss these findings within the context of parental investment theory and sexual strategies theory.

Reducing the Bias of Norm Scores in Non-Representative Samples: Weighting as an Adjunct to Continuous Norming Methods.

We investigated whether the accuracy of normed test scores derived from non-demographically representative samples can be improved by combining continuous norming methods with compensatory weighting of test results. To this end, we introduce Raking, a method from social sciences, to psychometrics. In a simulated reference population, we modeled a latent cognitive ability with a typical developmental gradient, along with three demographic variables that were correlated to varying degrees with the latent ability. We simulated five additional populations representing patterns of non-representativeness that might be encountered in the real world. We subsequently drew smaller normative samples from each population and used an one-parameter logistic Item Response Theory (IRT) model to generate simulated test results for each individual. Using these simulated data, we applied norming techniques, both with and without compensatory weighting. Weighting reduced the bias of the norm scores when the degree of non-representativeness was moderate, with only a small risk of generating new biases.

Accounting for Intraindividual Profiles in the Wechsler Intelligence Scales Improves the Prediction of School Performance.

IQ scores are often used to predict school performance. However, for children with learning disabilities, the predictive validity of IQ scores appears to be low. In the fourth and fifth versions of the Wechsler Intelligence Scales for Children (WISC), new ancillary indices were introduced. The General Ability Index (GAI) is a broad measure of fluid reasoning with verbal, visual-spatial, and figural test items. By contrast, the Cognitive Proficiency Index (CPI) combines different executive functions known to be frequently affected in children with dyslexia, ADHD, or combined learning disorders. To date, there is little evidence to demonstrate that these measures improve the prediction of school performance beyond the Full-scale IQ (FSIQ). We therefore used lasso regression to explore the predictive validity of these measures for school grades. The analyzed samples were taken from the German standardization samples of the WISC-IV and the WISC-V. In most cases, the prediction of school performance was not considerably improved by taking the GAI or the CPI into account. However, when the individual discrepancy between the CPI and the GAI was high, the FSIQ lost its predictive validity in elementary school. In this subgroup of children, reading and writing skills were best predicted by the CPI, whereas math skills were limited by the lowest score (i.e., the minimum out of the CPI and the GAI).

Improvement of Norm Score Quality via Regression-Based Continuous Norming.

The interpretation of psychometric test results is usually based on norm scores. We compared semiparametric continuous norming (SPCN) with conventional norming methods by simulating results for test scales with different item numbers and difficulties via an item response theory approach. Subsequently, we modeled the norm scores based on random samples with varying sizes either with a conventional ranking procedure or SPCN. The norms were then cross-validated by using an entirely representative sample of N = 840,000 for which different measures of norming error were computed. This process was repeated 90,000 times. Both approaches benefitted from an increase in sample size, with SPCN reaching optimal results with much smaller samples. Conventional norming performed worse on data fit, age-related errors, and number of missings in the norm tables. The data fit in conventional norming of fixed subsample sizes varied with the granularity of the age brackets, calling into question general recommendations for sample sizes in test norming. We recommend that test norms should be based on statistical models of the raw score distributions instead of simply compiling norm tables via conventional ranking procedures.

Continuous norming of psychometric tests: A simulation study of parametric and semi-parametric approaches.

Continuous norming methods have seldom been subjected to scientific review. In this simulation study, we compared parametric with semi-parametric continuous norming methods in psychometric tests by constructing a fictitious population model within which a latent ability increases with age across seven age groups. We drew samples of different sizes (n = 50, 75, 100, 150, 250, 500 and 1,000 per age group) and simulated the results of an easy, medium, and difficult test scale based on Item Response Theory (IRT). We subjected the resulting data to different continuous norming methods and compared the data fit under the different test conditions with a representative cross-validation dataset of n = 10,000 per age group. The most significant differences were found in suboptimal (i.e., too easy or too difficult) test scales and in ability levels that were far from the population mean. We discuss the results with regard to the selection of the appropriate modeling techniques in psychometric test construction, the required sample sizes, and the requirement to report appropriate quantitative and qualitative test quality criteria for continuous norming methods in test manuals.

A Continuous Solution to the Norming Problem.

Conventional methods for producing test norms are often plagued with "jumps" or "gaps" (i.e., discontinuities) in norm tables and low confidence for assessing extreme scores. We propose a new approach for producing continuous test norms to address these problems that also has the added advantage of not requiring assumptions about the distribution of the raw data: Norm values are established from raw data by modeling the latter ones as a function of both percentile scores and an explanatory variable (e.g., age). The proposed method appears to minimize bias arising from sampling and measurement error, while handling marked deviations from normality-such as are commonplace in clinical samples. In addition to step-by-step instructions in how to apply this method, we demonstrate its advantages over conventional discrete norming procedures using norming data from two different psychometric tests, employing either age norms ( N = 3,555) or grade norms ( N = 1,400).

Movements or targets: what makes an action in action-effect learning?

According to ideomotor theory, actions become linked to the sensory feedback they contingently produce, so that anticipating the feedback automatically evokes the action it typically results from. Numerous recent studies have provided evidence in favour of such action-effect learning but left an important issue unresolved. It remains unspecified to what extent action-effect learning is based on associating effect-representations to representations of the performed movements or to representations of the targets at which the behaviour aimed at. Two experiments were designed to clarify this issue. In an acquisition phase, participants learned the contingency between key presses and effect tones. In a following test phase, key-effect and movement-effect relations were orthogonally assessed by changing the hand-key mapping for one half of the participants. Experiment 1 showed precedence for target-effect over movement-effect learning in a forced-choice RT task. In Experiment 2, target-effect learning was also shown to influence the outcome of response selection in a free-choice task. Altogether, the data indicate that both movement-effect and target-effect associations contribute to the formation of action-effect linkages-provided that movements and targets are likewise contingently related to the effects.

Explorations of anticipatory behavioral control (ABC): a report from the cognitive psychology unit of the University of Würzburg.

The report comprises recent theoretical considerations, experimental research, and simulations which all aim at a clarification of anticipatory mechanisms of behavioral control.

Constant error in aiming movements without visual feedback is higher in the preferred hand.

There is convincing evidence for a left hand advantage for the spatial planning of aiming movements in right-handers. However, little is known about equivalent proficiency in left-handers. Therefore, 48 participants (24 right-handers and 24 left-handers) performed aiming movements of the hand without visual feedback. While the variable aiming error tended to be lower for the preferred hand, the constant aiming error was consistently lower for the non-preferred hand. Data are consistent with the idea of a spatial accuracy advantage for the controller of the non-preferred hand. Data from an ambidextrous participant suggest that this functional difference might be innate rather than acquired through practice.

Psychology has to deal with efferences too.

The authors argue that F. Mechsner's (2004) perceptual-cognitive approach is an important contribution to the current discussion of motor control. Mechsner presents convincing evidence against direct motor control. Nevertheless, the mechanisms by which perceptual-cognitive representations determine efferent commands must still be clarified.