Problem solving is embedded in context so how do we measure it?

Problem solving encompasses the broad domain of human, goal-directed behaviors. Though we may attempt to measure problem solving using tightly controlled and decontextualized tasks, it is inextricably embedded in both reasoners' experiences and their contexts. Without situating problem solvers, problem contexts, and our own experiential partialities as researchers, we risk intertwining the research of information relevance with our own confirmatory biases about people, environments, and ourselves. We review each of these ecological facets of information relevance in problem solving, and we suggest a framework to guide its measurement. We ground this framework with concrete examples of ecologically valid, culturally relevant measurement of problem solving.

The Growth of Complex Syntax in School-Age African American Children Who Speak African American English.

PURPOSE: Syntax provides critical support for both academic success and linguistic growth, yet it has not been a focus of language research in school-age African American children. This study examines complex syntax performance of African American children in second through fifth grades. METHOD: The current study explores the syntactic performances of African American children (N = 513) in Grades 2-5 on the Test of Language Development-Intermediate who speak African American English. Multilevel modeling was used to evaluate the growth and associated changes between dialect density and syntax. Analyzed data were compared both to the normative sample and within the recruited sample. RESULTS: The results suggest that dialect density exerted its impact early but did not continue to influence syntactic growth over time. Additionally, it was not until dialect density was accounted for in growth models that African American children's syntactic growth resembled normative expectations of a standardized language instrument. CONCLUSION: The current study suggests that failure to consider cultural language differences obscures our understanding of African American students' linguistic competence on standardized language assessments.

Developing a Longitudinal Scale for Language: Linking Across Developmentally Different Versions of the Same Test.

Purpose Many language tests use different versions that are not statistically linked or do not have a developmental scaled score. The current article illustrates the problems of scores that are not linked or equated, followed by a statistical model to derive a developmental scaled score. Method Using an accelerated cohort design of 890 students in Grades 1-5, a confirmatory factor model was fit to 6 subtests of the Test of Language Development-Primary and Intermediate: Fourth Edition ( Hammill & Newcomer, 2008a , 2008b ). The model allowed for linking the subtests to a general factor of language and equating their measurement characteristics across grades and cohorts of children. A sequence of models was fit to evaluate the appropriateness of the linking assumptions. Results The models fit well, with reasonable support for the validity of the tests to measure a general factor of language on a longitudinally consistent scale. Conclusion Although total and standard scores were problematic for longitudinal relations, the results of the model suggest that language grows in a relatively linear manner among these children, regardless of which set of subtests they received. Researchers and clinicians interested in longitudinal inferences are advised to design research or choose tests that can provide a developmental scaled score.

Measuring arithmetic: A psychometric approach to understanding formatting effects and domain specificity.

Using multitrait, multimethod data, and confirmatory factor analysis, the current study examined the effects of arithmetic item formatting and the possibility that across formats, abilities other than arithmetic may contribute to children's answers. Measurement hypotheses were guided by several leading theories of arithmetic cognition. With a sample of 1314 3rd grade students (age M=103.24 months, SD=5.41 months), Abstract Code Theory, Encoding Complex Theory, Triple Code Theory, and the Exact versus Approximate Calculations Hypothesis were evaluated, using 11 measures of arithmetic with symbolic problem formats (e.g., Arabic numeral and language-based formats) and various problem demands (e.g., requiring both exact and approximate calculations). In general, results provided support for both Triple Code Theory and Encoding Complex Theory. As predicted by Triple Code Theory, arithmetic outcomes with language formatting, Arabic numeral formatting, and estimation demands (across formats) were related but distinct from one another. As predicted by Encoding Complex Theory, executive attention was a direct predictor of all arithmetic outcomes. Language was no longer a direct predictor of arithmetic outcomes when executive attention was accounted for in the model; however, a strong and enduring relationship between language and executive attention suggested that language may play a facilitative role in reasoning during numeric processing. These findings have important implications for assessing arithmetic in educational settings and suggest that in addition to arithmetic-focused interventions, interventions targeting executive attention, language, and/or the interplay between them (i.e., internal speech during problem-solving) may be a promising avenues of mathematical problem-solving intervention.

Testing Math or Testing Language? The Construct Validity of the KeyMath-Revised for Children With Intellectual Disability and Language Difficulties.

Although it is often assumed that mathematics ability alone predicts mathematics test performance, linguistic demands may also predict achievement. This study examined the role of language in mathematics assessment performance for children with intellectual disability (ID) at less severe levels, on the KeyMath-Revised Inventory (KM-R) with a sample of 264 children, in grades 2-5. Using confirmatory factor analysis, the hypothesis that the KM-R would demonstrate discriminant validity with measures of language abilities in a two-factor model was compared to two plausible alternative models. Results indicated that KM-R did not have discriminant validity with measures of children's language abilities and was a multidimensional test of both mathematics and language abilities for this population of test users. Implications are considered for test development, interpretation, and intervention.