A Novel Approach to Measuring an Old Construct: Aligning the Conceptualisation and Operationalisation of Cognitive Flexibility.

A successful adjustment to dynamic changes in one's environment requires contingent adaptive behaviour. Such behaviour is underpinned by cognitive flexibility, which conceptually is part of fluid intelligence. We argue, however, that conventional approaches to measuring fluid intelligence are insufficient in capturing cognitive flexibility. We address the discrepancy between conceptualisation and operationalisation by introducing two newly developed tasks that aim at capturing within-person processes of dealing with novelty. In an exploratory proof-of-concept study, the two flexibility tasks were administered to 307 university students, together with a battery of conventional measures of fluid intelligence. Participants also provided information about their Grade Point Averages obtained in high school and in their first year at university. We tested (1) whether an experimental manipulation of a requirement for cognitive inhibition resulted in systematic differences in difficulty, (2) whether these complexity differences reflect psychometrically differentiable effects, and (3) whether these newly developed flexibility tasks show incremental value in predicting success in the transition from high school to university over conventional operationalisations of fluid intelligence. Our findings support the notion that cognitive flexibility, when conceptualised and operationalised as individual differences in within-person processes of dealing with novelty, more appropriately reflects the dynamics of individuals' behaviour when attempting to cope with changing demands.

Intelligence IS Cognitive Flexibility: Why Multilevel Models of Within-Individual Processes Are Needed to Realise This.

Despite substantial evidence for the link between an individual's intelligence and successful life outcomes, questions about what defines intelligence have remained the focus of heated dispute. The most common approach to understanding intelligence has been to investigate what performance on tests of intellect is and is not associated with. This psychometric approach, based on correlations and factor analysis is deficient. In this review, we aim to substantiate why classic psychometrics which focus on between-person accounts will necessarily provide a limited account of intelligence until theoretical considerations of within-person accounts are incorporated. First, we consider the impact of entrenched psychometric presumptions that support the status quo and impede alternative views. Second, we review the importance of process-theories, which are critical for any serious attempt to build a within-person account of intelligence. Third, features of dynamic tasks are reviewed, and we outline how static tasks can be modified to target within-person processes. Finally, we explain how multilevel models are conceptually and psychometrically well-suited to building and testing within-individual notions of intelligence, which at its core, we argue is cognitive flexibility. We conclude by describing an application of these ideas in the context of microworlds as a case study.

Sophisticated Statistics Cannot Compensate for Method Effects If Quantifiable Structure Is Compromised.

Researchers rely on psychometric principles when trying to gain understanding of unobservable psychological phenomena disconfounded from the methods used. Psychometric models provide us with tools to support this endeavour, but they are agnostic to the meaning researchers intend to attribute to the data. We define method effects as resulting from actions which weaken the psychometric structure of measurement, and argue that solution to this confounding will ultimately rest on testing whether data collected fit a psychometric model based on a substantive theory, rather than a search for a model that best fits the data. We highlight the importance of taking the notions of fundamental measurement seriously by reviewing distinctions between the Rasch measurement model and more generalised 2PL and 3PL IRT models. We then present two lines of research that highlight considerations of making method effects explicit in experimental designs. First, we contrast the use of experimental manipulations to study measurement reactivity during the assessment of metacognitive processes with factor-analytic research of the same. The former suggests differential performance-facilitating and -inhibiting reactivity as a function of other individual differences, whereas factor-analytic research suggests a ubiquitous monotonically predictive confidence factor. Second, we evaluate differential effects of context and source on within-individual variability indices of personality derived from multiple observations, highlighting again the importance of a structured and theoretically grounded observational framework. We conclude by arguing that substantive variables can act as method effects and should be considered at the time of design rather than after the fact, and without compromising measurement ideals.

Manifesto for new directions in developmental science.

Although developmental science has always been evolving, these times of fast-paced and profound social and scientific changes easily lead to disorienting fragmentation rather than coherent scientific advances. What directions should developmental science pursue to meaningfully address real-world problems that impact human development throughout the lifespan? What conceptual or policy shifts are needed to steer the field in these directions? The present manifesto is proposed by a group of scholars from various disciplines and perspectives within developmental science to spark conversations and action plans in response to these questions. After highlighting four critical content domains that merit concentrated and often urgent research efforts, two issues regarding "how" we do developmental science and "what for" are outlined. This manifesto concludes with five proposals, calling for integrative, inclusive, transdisciplinary, transparent, and actionable developmental science. Specific recommendations, prospects, pitfalls, and challenges to reach this goal are discussed.

Deferential Trespassing: Looking Through and at an Intersectional Lens.

In this article, I comment on the prospect of integrating an intersectionality perspective into the developmental sciences. I do this by sharing impressions, insights, and questions that have emerged whilst attempting to look at and to look through an intersectionality lens. My comments focus on three main topics. First, I speculate what forms such an integration could take and argue that an integration that productively contributes to shaping developmental science into a transdisciplinary field is likely to change intersectionality research itself. I then reflect on the perceived ambiguity in terms of the unit of analysis (e.g., social systems vs. individuals) and the focus of research questions (i.e., description vs. explanation vs. intervention) in intersectionality research. Clarity and transparency in this regard is instrumental to productively identifying conceptual and methodological overlaps or intersections with other subdisciplines in developmental sciences. Finally, I highlight the importance of development being more comprehensively reflected in the conceptualizations, the research questions, and the subsequently employed methodologies in intersectionality research. I conclude with a plea for allowing our expertise to intersect to transdisciplinarily work toward creating systemic and perpetual progress in the developmental sciences-something, I believe to resonate strongly with intersectionality researchers.

Beyond Psychometrics: The Difference between Difficult Problem Solving and Complex Problem Solving.

In this paper we argue that a synthesis of findings across the various sub-areas of research in complex problem solving and consequently progress in theory building is hampered by an insufficient differentiation of complexity and difficulty. In the proposed framework of person, task, and situation (PTS), complexity is conceptualized as a quality that is determined by the cognitive demands that the characteristics of the task and the situation impose. Difficulty represents the quantifiable level of a person's success in dealing with such demands. We use the well-documented "semantic effect" as an exemplar for testing some of the conceptual assumptions derived from the PTS framework. We demonstrate how a differentiation between complexity and difficulty can help take beyond a potentially too narrowly defined psychometric perspective and subsequently gain a better understanding of the cognitive mechanisms behind this effect. In an empirical study a total of 240 university students were randomly allocated to one of four conditions. The four conditions resulted from contrasting the semanticity level of the variable labels used in the CPS system (high vs. low) and two instruction conditions for how to explore the CPS system's causal structure (starting with the assumption that all relationships between variables existed vs. starting with the assumption that none of the relationships existed). The variation in the instruction aimed at inducing knowledge acquisition processes of either (1) systematic elimination of presumptions, or (2) systematic compilation of a mental representation of the causal structure underpinning the system. Results indicate that (a) it is more complex to adopt a "blank slate" perspective under high semanticity as it requires processes of inhibiting prior assumptions, and (b) it seems more difficult to employ a systematic heuristic when testing against presumptions. In combination, situational characteristics, such as the semanticity of variable labels, have the potential to trigger qualitatively different tasks. Failing to differentiate between 'task' and 'situation' as independent sources of complexity and treating complexity and difficulty synonymously threaten the validity of performance scores obtained in CPS research.

Missing the Wood for the Wrong Trees: On the Difficulty of Defining the Complexity of Complex Problem Solving Scenarios.

In this paper we discuss how the lack of a common framework in Complex Problem Solving (CPS) creates a major hindrance to a productive integration of findings and insights gained in its 40+-year history of research. We propose a framework that anchors complexity within the tri-dimensional variable space of Person, Task and Situation. Complexity is determined by the number of information cues that need to be processed in parallel. What constitutes an information cue is dependent on the kind of task, the system or CPS scenario used and the task environment (i.e., situation) in which the task is performed. Difficulty is conceptualised as a person's subjective reflection of complexity. Using an existing data set of N = 294 university students' problem solving performances, we test the assumption derived from this framework that particular system features such as numbers of variables (NoV) or numbers of relationships (NoR) are inappropriate indicators of complexity. We do so by contrasting control performance across four systems that differ in these attributes. Results suggest that for controlling systems (task) with semantically neutral embedment (situation), the maximum number of dependencies any of the output variables has is a promising indicator of this task's complexity.

Bayesian Analysis of Individual Level Personality Dynamics.

A Bayesian technique with analyses of within-person processes at the level of the individual is presented. The approach is used to examine whether the patterns of within-person responses on a 12-trial simulation task are consistent with the predictions of ITA theory (Dweck, 1999). ITA theory states that the performance of an individual with an entity theory of ability is more likely to spiral down following a failure experience than the performance of an individual with an incremental theory of ability. This is because entity theorists interpret failure experiences as evidence of a lack of ability which they believe is largely innate and therefore relatively fixed; whilst incremental theorists believe in the malleability of abilities and interpret failure experiences as evidence of more controllable factors such as poor strategy or lack of effort. The results of our analyses support ITA theory at both the within- and between-person levels of analyses and demonstrate the benefits of Bayesian techniques for the analysis of within-person processes. These include more formal specification of the theory and the ability to draw inferences about each individual, which allows for more nuanced interpretations of individuals within a personality category, such as differences in the individual probabilities of spiraling. While Bayesian techniques have many potential advantages for the analyses of processes at the level of the individual, ease of use is not one of them for psychologists trained in traditional frequentist statistical techniques.

Commentary-of quadrants and fish scales: reflections on new directions in research in child and adolescent development.

In this article, I reflect on how ways of reporting research as well as reviewing and commenting on submitted manuscripts could take new directions to promote progress in the discipline of developmental science. I argue for (a) attitudinal openness toward migratory impulses in relation to Stokes's quadrant model of science, (b) the relinquishment of an unproductive sense of problem ownership in authors, reviewers, and commentators, (c) active attempts to fill intra- and interdisciplinary gaps rather than solely focusing on strengthening existing islets of disciplinary expertise, (d) a strategic diversification of expertise in the selection of reviewers and commentators, (e) adopting a communication style that can be described as deferential transgressions, and (f) promoting attempts for transdisciplinary replications. © 2015 Wiley Periodicals, Inc.

Intelligence and individual differences in becoming neurally efficient.

Physiological approaches to human psychometric intelligence have shown a higher neural efficiency (i.e. less cortical activation) during cognitive performance in brighter subjects. The main aim of this study was to explore the relationship between intelligence and cortical activation patterns in the framework of the learning test concept. In 27 participants we assessed the topography and extent of cortical activation by means of event-related desynchronization (ERD) during reasoning tests in a pre-test--training--post-test design and related it to psychometric intelligence (measured by the German Leistungs-Prüf-System, LPS). Significant associations between intelligence and cortical activation patterns were exclusively found at anterior (frontal) recording sites, which corroborates the central role of the frontal lobe for higher-order cognitive functions. The hypothesized negative intelligence-activation correlation was observed only after the training, i.e. in the post-test, but not in the pre-test. More important, the decrease in cortical investment from pre-test to post-test correlated negatively with intelligence, indicating that the higher the subjects' general mental ability the larger the decrease in the amount of cortical activation. These findings suggest intelligence-related individual differences in becoming neurally efficient.