Disembodied AI and the limits to machine understanding of students' embodied interactions.

The embodiment turn in the Learning Sciences has fueled growth of multimodal learning analytics to understand embodied interactions and make consequential educational decisions about students more rapidly, more accurately, and more personalized than ever before. Managing demands of complexity and speed is leading to growing reliance by education systems on disembodied artificial intelligence (dAI) programs, which, ironically, are inherently incapable of interpreting students' embodied interactions. This is fueling a potential crisis of complexity. Augmented intelligence systems offer promising avenues for managing this crisis by integrating the strengths of omnipresent dAI to detect complex patterns of student behavior from multimodal datastreams, with the strengths of humans to meaningfully interpret embodied interactions in service of consequential decision making to achieve a balance between complexity, interpretability, and accountability for allocating education resources to children.

The Effect of Cognitive Relevance of Directed Actions on Mathematical Reasoning.

Theories of grounded and embodied cognition offer a range of accounts of how reasoning and body-based processes are related to each other. To advance theories of grounded and embodied cognition, we explore the cognitive relevance of particular body states to associated math concepts. We test competing models of action-cognition transduction to investigate the cognitive relevance of directed actions to students' mathematical reasoning in the area of geometry. The hypotheses we test include (1) that cognitively relevant directed actions have a direct effect on performance (direct cognitive relevance hypothesis), (2) that cognitively relevant directed actions lead to more frequent production of gestures during explanations, which leads to improved performance (mediated cognitive relevance hypothesis), and (3) that performance effects of directed actions are influenced by the presence or absence of gesture production during mathematical explanations (moderated cognitive relevance hypothesis). We explore these hypotheses in an experiment where high school students (N = 85) evaluated the truth of geometry conjectures after performing cognitively relevant or cognitively irrelevant directed actions while playing a movement-based video game. Contrary to the direct and mediated cognitive relevance hypotheses, we found no overall differences in performance or gesture production between relevant and irrelevant conditions. Consistent with the moderated cognitive relevance hypothesis, cognitive relevance influenced mathematical performance, as measured by the accuracy of students' intuitions, insights, and the validity of their proofs, provided that students produced certain kinds of gestures during mathematical explanations (i.e., with explanatory gestures as the moderator). Implications for theories of grounded and embodied cognition and the design of embodied forms of educational interventions are discussed.

Low agreement among reviewers evaluating the same NIH grant applications.

Obtaining grant funding from the National Institutes of Health (NIH) is increasingly competitive, as funding success rates have declined over the past decade. To allocate relatively scarce funds, scientific peer reviewers must differentiate the very best applications from comparatively weaker ones. Despite the importance of this determination, little research has explored how reviewers assign ratings to the applications they review and whether there is consistency in the reviewers' evaluation of the same application. Replicating all aspects of the NIH peer-review process, we examined 43 individual reviewers' ratings and written critiques of the same group of 25 NIH grant applications. Results showed no agreement among reviewers regarding the quality of the applications in either their qualitative or quantitative evaluations. Although all reviewers received the same instructions on how to rate applications and format their written critiques, we also found no agreement in how reviewers "translated" a given number of strengths and weaknesses into a numeric rating. It appeared that the outcome of the grant review depended more on the reviewer to whom the grant was assigned than the research proposed in the grant. This research replicates the NIH peer-review process to examine in detail the qualitative and quantitative judgments of different reviewers examining the same application, and our results have broad relevance for scientific grant peer review.

Teachers' gestures and students' learning: sometimes "hands off" is better.

During mathematics instruction, teachers often make links between different representations of mathematical information, and they sometimes use gestures to refer to the representations that they link. In this research, we investigated the role of such gestures in students' learning from lessons about links between linear equations and corresponding graphs. Eighty-two middle-school students completed a pretest, viewed a video lesson, and then completed a posttest comparable to the pretest. In all of the video lessons, the teacher explained the links between equations and graphs in speech. The lessons varied in whether the teacher referred to the equations in gesture and in whether she referred to the graphs in gesture, yielding four conditions: neither equations nor graphs, equations only, graphs only, and both equations and graphs. In all conditions, the gestures were redundant with speech, in the sense that the referents of the gestures were also mentioned in speech (e.g., pointing to "2" while saying "2"). Students showed substantial learning in all conditions. However, students learned less when the teacher referred to the equations in gesture than when she did not. This was not the case for gesture to graphs. These findings are discussed in terms of the processing implications of redundancy between gesture and speech, and the possibility of "trade-offs" in attention to the visual representations. The findings underscore the need for a more nuanced view of the role of teachers' gestures in students' comprehension and learning.

'Your comments are meaner than your score': score calibration talk influences intra- and inter-panel variability during scientific grant peer review.

In scientific grant peer review, groups of expert scientists meet to engage in the collaborative decision-making task of evaluating and scoring grant applications. Prior research on grant peer review has established that inter-reviewer reliability is typically poor. In the current study, experienced reviewers for the National Institutes of Health (NIH) were recruited to participate in one of four constructed peer review panel meetings. Each panel discussed and scored the same pool of recently reviewed NIH grant applications. We examined the degree of intra-panel variability in panels' scores of the applications before versus after collaborative discussion, and the degree of inter-panel variability. We also analyzed videotapes of reviewers' interactions for instances of one particular form of discourse-Score Calibration Talk-as one factor influencing the variability we observe. Results suggest that although reviewers within a single panel agree more following collaborative discussion, different panels agree less after discussion, and Score Calibration Talk plays a pivotal role in scoring variability during peer review. We discuss implications of this variability for the scientific peer review process.

Grounded and embodied mathematical cognition: Promoting mathematical insight and proof using action and language.

We develop a theory of grounded and embodied mathematical cognition (GEMC) that draws on action-cognition transduction for advancing understanding of how the body can support mathematical reasoning. GEMC proposes that participants' actions serve as inputs capable of driving the cognition-action system toward associated cognitive states. This occurs through a process of transduction that promotes valuable mathematical insights by eliciting dynamic depictive gestures that enact spatio-temporal properties of mathematical entities. Our focus here is on pre-college geometry proof production. GEMC suggests that action alone can foster insight but is insufficient for valid proof production if action is not coordinated with language systems for propositionalizing general properties of objects and space. GEMC guides the design of a video game-based learning environment intended to promote students' mathematical insights and informal proofs by eliciting dynamic gestures through in-game directed actions. GEMC generates several hypotheses that contribute to theories of embodied cognition and to the design of science, technology, engineering, and mathematics (STEM) education interventions. Pilot study results with a prototype video game tentatively support theory-based predictions regarding the role of dynamic gestures for fostering insight and proof-with-insight, and for the role of action coupled with language to promote proof-with-insight. But the pilot yields mixed results for deriving in-game interventions intended to elicit dynamic gesture production. Although our central purpose is an explication of GEMC theory and the role of action-cognition transduction, the theory-based video game design reveals the potential of GEMC to improve STEM education, and highlights the complex challenges of connecting embodiment research to education practices and learning environment design.

Improving Students' Learning With Effective Learning Techniques: Promising Directions From Cognitive and Educational Psychology.

Many students are being left behind by an educational system that some people believe is in crisis. Improving educational outcomes will require efforts on many fronts, but a central premise of this monograph is that one part of a solution involves helping students to better regulate their learning through the use of effective learning techniques. Fortunately, cognitive and educational psychologists have been developing and evaluating easy-to-use learning techniques that could help students achieve their learning goals. In this monograph, we discuss 10 learning techniques in detail and offer recommendations about their relative utility. We selected techniques that were expected to be relatively easy to use and hence could be adopted by many students. Also, some techniques (e.g., highlighting and rereading) were selected because students report relying heavily on them, which makes it especially important to examine how well they work. The techniques include elaborative interrogation, self-explanation, summarization, highlighting (or underlining), the keyword mnemonic, imagery use for text learning, rereading, practice testing, distributed practice, and interleaved practice. To offer recommendations about the relative utility of these techniques, we evaluated whether their benefits generalize across four categories of variables: learning conditions, student characteristics, materials, and criterion tasks. Learning conditions include aspects of the learning environment in which the technique is implemented, such as whether a student studies alone or with a group. Student characteristics include variables such as age, ability, and level of prior knowledge. Materials vary from simple concepts to mathematical problems to complicated science texts. Criterion tasks include different outcome measures that are relevant to student achievement, such as those tapping memory, problem solving, and comprehension. We attempted to provide thorough reviews for each technique, so this monograph is rather lengthy. However, we also wrote the monograph in a modular fashion, so it is easy to use. In particular, each review is divided into the following sections: General description of the technique and why it should work How general are the effects of this technique? 2a. Learning conditions 2b. Student characteristics 2c. Materials 2d. Criterion tasks Effects in representative educational contexts Issues for implementation Overall assessment The review for each technique can be read independently of the others, and particular variables of interest can be easily compared across techniques. To foreshadow our final recommendations, the techniques vary widely with respect to their generalizability and promise for improving student learning. Practice testing and distributed practice received high utility assessments because they benefit learners of different ages and abilities and have been shown to boost students' performance across many criterion tasks and even in educational contexts. Elaborative interrogation, self-explanation, and interleaved practice received moderate utility assessments. The benefits of these techniques do generalize across some variables, yet despite their promise, they fell short of a high utility assessment because the evidence for their efficacy is limited. For instance, elaborative interrogation and self-explanation have not been adequately evaluated in educational contexts, and the benefits of interleaving have just begun to be systematically explored, so the ultimate effectiveness of these techniques is currently unknown. Nevertheless, the techniques that received moderate-utility ratings show enough promise for us to recommend their use in appropriate situations, which we describe in detail within the review of each technique. Five techniques received a low utility assessment: summarization, highlighting, the keyword mnemonic, imagery use for text learning, and rereading. These techniques were rated as low utility for numerous reasons. Summarization and imagery use for text learning have been shown to help some students on some criterion tasks, yet the conditions under which these techniques produce benefits are limited, and much research is still needed to fully explore their overall effectiveness. The keyword mnemonic is difficult to implement in some contexts, and it appears to benefit students for a limited number of materials and for short retention intervals. Most students report rereading and highlighting, yet these techniques do not consistently boost students' performance, so other techniques should be used in their place (e.g., practice testing instead of rereading). Our hope is that this monograph will foster improvements in student learning, not only by showcasing which learning techniques are likely to have the most generalizable effects but also by encouraging researchers to continue investigating the most promising techniques. Accordingly, in our closing remarks, we discuss some issues for how these techniques could be implemented by teachers and students, and we highlight directions for future research.

Learning sciences.

The aims of the learning sciences (LS) are to understand the nature of learning from a broad range of perspectives, and to shape the ways that learning environments and resources are designed and used. LS incorporates both systemic and elemental approaches to investigating questions about learning, as a complement to the primarily elemental approach emphasized in cognitive science research. Thus, its greatest potential is in the integration of systemic and elemental perspectives. Four major themes are central. First, research in LS attempts to bridge the divide between research and practice. Second, research in LS is motivated by limitations of theories of learning and cognition for specifying instruction. Third, research in LS embraces the importance of analyzing and assessing complex interventions through both experimental and design-based research. Fourth, research in LS emphasizes the learning and behavior of the individual in interaction with the physical, social, and cultural world, as well as with semiotic and technical resources. Research in LS can be conceptualized along a continuum of time scales, from the more microscopic to the more macroscopic. The time-scale framework illustrates how disparate research traditions and research methods can function within a unifying framework for the study of learning and complex behavior. The effort to 'scale-up' from more elemental findings to more complex, authentic settings has been generative for LS, but faces serious challenges. There is an alternate route to establishing a cumulative scientific knowledge base, namely, 'scaling down' from more complex, ecologically valid levels to more elemental levels. Studies of basic learning processes, framed in the context of the larger system, are well positioned to support impact in authentic settings. Copyright © 2010 John Wiley & Sons, Ltd. For further resources related to this article, please visit the WIREs website.

Trade-offs between grounded and abstract representations: evidence from algebra problem solving.

This article explores the complementary strengths and weaknesses of grounded and abstract representations in the domain of early algebra. Abstract representations, such as algebraic symbols, are concise and easy to manipulate but are distanced from any physical referents. Grounded representations, such as verbal descriptions of situations, are more concrete and familiar, and they are more similar to physical objects and everyday experience. The complementary computational characteristics of grounded and abstract representations lead to trade-offs in problem-solving performance. In prior research with high school students solving relatively simple problems, Koedinger and Nathan (2004) demonstrated performance benefits of grounded representations over abstract representations-students were better at solving simple story problems than the analogous equations. This article extends this prior work to examine both simple and more complex problems in two samples of college students. On complex problems with two references to the unknown, a "symbolic advantage" emerged, such that students were better at solving equations than analogous story problems. Furthermore, the previously observed "verbal advantage" on simple problems was replicated. We thus provide empirical support for a trade-off between grounded, verbal representations, which show advantages on simpler problems, and abstract, symbolic representations, which show advantages on more complex problems.