How Prior Knowledge, Gesture Instruction, and Interference After Instruction Interact to Influence Learning of Mathematical Equivalence.

Although children learn more when teachers gesture, it is not clear how gesture supports learning. Here, we sought to investigate the nature of the memory processes that underlie the observed benefits of gesture on lasting learning. We hypothesized that instruction with gesture might create memory representations that are particularly resistant to interference. We investigated this possibility in a classroom study with 402 second- and third-grade children. Participants received classroom-level instruction in mathematical equivalence using videos with or without accompanying gesture. After instruction, children solved problems that were either visually similar to the problems that were taught, and consistent with an operational interpretation of the equal sign (interference), or visually distinct from equivalence problems and without an equal sign (control) in order to assess the role of gesture in resisting interference after learning. Gesture facilitated learning, but the effects of gesture and interference varied depending on type of problem being solved and the strategies that children used to solve problems prior to instruction. Some children benefitted from gesture, while others did not. These findings have implications for understanding the mechanisms underlying the beneficial effect of gesture on mathematical learning, revealing that gesture does not work via a general mechanism like enhancing attention or engagement that would apply to children with all forms of prior knowledge.

Emoji as gesture in digital communication: Emoji improve comprehension of indirect speech.

As reliance on digital communication grows, so does the importance of communicating effectively with text. Yet when communicating with text, benefits from other channels, such as hand gesture, are diminished. Hand gestures support comprehension and disambiguate characteristics of the spoken message by providing information in a visual channel supporting speech. Can emoji (pictures used to supplement text communication) perform similar functions? Here, we ask whether emoji improve comprehension of indirect speech. Indirect speech is ambiguous, and appropriate comprehension depends on the receiver decoding context cues, such as hand gesture. We adapted gesture conditions from prior research (Kelly et al., 1999, Experiment 2) to a digital, text-based format, using emoji rather than gestures. Participants interpreted 12 hypothetical text-message exchanges that ended with indirect speech, communicated via text only, text+emoji, or emoji only, in a between-subjects design. Like that previously seen for hand gesture, emoji improved comprehension. Participants were more likely to correctly interpret indirect speech in the emoji-only condition compared with the text+emoji and the text-only conditions, and more likely in the text+emoji condition compared to the text-only condition. Thus, emoji are not mere decoration, but rather are integrated with text to communicate and disambiguate complex messages. Similar to gesture in face-to-face communication, emoji improve comprehension during text-based communication.

Teaching little kids big sentences: A randomized controlled trial showing that children with DLD respond to complex syntax intervention embedded within the context of preschool/kindergarten science instruction.

BACKGROUND: The language of the science curriculum is complex, even in the early grades. To communicate their scientific observations, children must produce complex syntax, particularly complement clauses (e.g., I think it will float; We noticed that it vibrates). Complex syntax is often challenging for children with developmental language disorder (DLD), and thus their learning and communication of science may be compromised. AIMS: We asked whether recast therapy delivered in the context of a science curriculum led to gains in complement clause use and scientific content knowledge. To understand the efficacy of recast therapy, we compared changes in science and language knowledge in children who received treatment for complement clauses embedded in a first-grade science curriculum to two active control conditions (vocabulary + science, phonological awareness + science). METHODS & PROCEDURES: This 2-year single-site three-arm parallel randomized controlled trial was conducted in Delaware, USA. Children with DLD, not yet in first grade and with low accuracy on complement clauses, were eligible. Thirty-three 4-7-year-old children participated in the summers of 2018 and 2019 (2020 was cancelled due to COVID-19). We assigned participants to arms using 1:1:1 pseudo-random allocation (avoiding placing siblings together). The intervention consisted of 39 small-group sessions of recast therapy, robust vocabulary instruction or phonological awareness intervention during eight science units over 4 weeks, followed by two science units (1 week) taught without language intervention. Pre-/post-measures were collected 3 weeks before and after camp by unmasked assessors. OUTCOMES & RESULTS: Primary outcome measures were accuracy on a 20-item probe of complement clause production and performance on ten 10-item unit tests (eight science + language, two science only). Complete data were available for 31 children (10 grammar, 21 active control); two others were lost to follow-up. Both groups made similar gains on science unit tests for science + language content (pre versus post, d = 2.9, p < 0.0001; group, p = 0.24). The grammar group performed significantly better at post-test than the active control group (d = 2.5, p = 0.049) on complement clause probes and marginally better on science-only unit tests (d = 2.5, p = 0.051). CONCLUSIONS & IMPLICATIONS: Children with DLD can benefit from language intervention embedded in curricular content and learn both language and science targets taught simultaneously. Tentative findings suggest that treatment for grammar targets may improve academic outcomes. WHAT THIS PAPER ADDS: What is already known on the subject We know that recast therapy focused on morphology is effective but very time consuming. Treatment for complex syntax in young children has preliminary efficacy data available. Prior research provides mixed evidence as to children's ability to learn language targets in conjunction with other information. What this study adds This study provides additional data supporting the efficacy of intensive complex syntax recast therapy for children ages 4-7 with Developmental Language Disorder. It also provides data that children can learn language targets and science curricular content simultaneously. What are the clinical implications of this work? As SLPs, we have to talk about something to deliver language therapy; we should consider talking about curricular content. Recast therapy focused on syntactic frames is effective with young children.

A pre-registered, multi-lab non-replication of the action-sentence compatibility effect (ACE).

The Action-sentence Compatibility Effect (ACE) is a well-known demonstration of the role of motor activity in the comprehension of language. Participants are asked to make sensibility judgments on sentences by producing movements toward the body or away from the body. The ACE is the finding that movements are faster when the direction of the movement (e.g., toward) matches the direction of the action in the to-be-judged sentence (e.g., Art gave you the pen describes action toward you). We report on a pre-registered, multi-lab replication of one version of the ACE. The results show that none of the 18 labs involved in the study observed a reliable ACE, and that the meta-analytic estimate of the size of the ACE was essentially zero.

The Challenge of Rich Vocabulary Instruction for Children With Developmental Language Disorder.

Purpose The aims of the study were to explore responses of children with developmental language disorder (DLD) to rich vocabulary instruction and to identify potential factors that contribute to outcomes. Method Children with DLD participated in a language intervention embedded within a science camp. Using parent and clinician reports, standardized tests, probes, notes, and video, we derived descriptions of seven of the campers who received a vocabulary intervention that incorporated principles of rich instruction. We present them here as a case series. Results Five cases responded to the intervention with modest gains in Tier 2 science vocabulary and science knowledge. One case demonstrated no response, and another was unable to complete the intervention. The latter two cases presented with triple risks: DLD, executive function deficits, and stressors associated with poverty. In comparison, the best responder also lived in poverty and had DLD, but he had intact executive function, strengths in extant vocabulary, stronger knowledge of science, better engagement in the science and language intervention activities, and was older. Other factors that seemed to contribute to outcomes included the complexity of the word forms and dosage. Conclusions Translating research on rich instruction to clinical practice is challenging. This case series motivated hypotheses about the nature of the challenge and what to do about it, the primary one being that the modest success of rich vocabulary instruction for children with DLD is not a limitation of the approach itself but rather a reflection of the difficulty of delivering the intervention while tailoring the targets, approach, and dosage to the needs of individual children with DLD. Supplemental Material https://doi.org/10.23641/asha.13667699.

Gesture during math instruction specifically benefits learners with high visuospatial working memory capacity.

BACKGROUND: Characteristics of both teachers and learners influence mathematical learning. For example, when teachers use hand gestures to support instruction, students learn more than others who learn the same concept with only speech, and students with higher working memory capacity (WMC) learn more rapidly than those with lower WMC. One hypothesis for the effect of gesture on math learning is that gestures provide a signal to learners that can reduce demand on working memory resources during learning. However, it is not known what sort of working memory resources support learning with gesture. Gestures are motoric; they co-occur with verbal language and they are perceived visually. METHODS: In two studies, we investigated the relationship between mathematical learning with or without gesture and individual variation in verbal, visuospatial, and kinesthetic WMC. Students observed a videotaped lesson in a novel mathematical system that either included instruction with both speech and gesture (Study 1) or instruction with only speech (Study 2). After instruction, students solved novel problems in the instructed system and transfer problems in a related system. Finally, students completed verbal, visuospatial, and kinesthetic working memory assessments. RESULTS: There was a positive relationship between visuospatial WMC and math learning when gesture was present, but no relationship between visuospatial WMC and math learning when gesture was absent. Rather, when gesture was absent, there was a relationship between verbal WMC and math learning. CONCLUSION: Providing gesture during instruction appears to change the cognitive resources recruited when learning a novel math task.

Patients with hippocampal amnesia successfully integrate gesture and speech.

During conversation, people integrate information from co-speech hand gestures with information in spoken language. For example, after hearing the sentence, "A piece of the log flew up and hit Carl in the face" while viewing a gesture directed at the nose, people tend to later report that the log hit Carl in the nose (information only in gesture) rather than in the face (information in speech). The cognitive and neural mechanisms that support the integration of gesture with speech are unclear. One possibility is that the hippocampus - known for its role in relational memory and information integration - is necessary for integrating gesture and speech. To test this possibility, we examined how patients with hippocampal amnesia and healthy and brain-damaged comparison participants express information from gesture in a narrative retelling task. Participants watched videos of an experimenter telling narratives that included hand gestures that contained supplementary information. Participants were asked to retell the narratives and their spoken retellings were assessed for the presence of information from gesture. For features that had been accompanied by supplementary gesture, patients with amnesia retold fewer of these features overall and fewer retellings that matched the speech from the narrative. Yet their retellings included features that contained information that had been present uniquely in gesture in amounts that were not reliably different from comparison groups. Thus, a functioning hippocampus is not necessary for gesture-speech integration over short timescales. Providing unique information in gesture may enhance communication for individuals with declarative memory impairment, possibly via non-declarative memory mechanisms.

Hand gestures support word learning in patients with hippocampal amnesia.

Co-speech hand gesture facilitates learning and memory, yet the cognitive and neural mechanisms supporting this remain unclear. One possibility is that motor information in gesture may engage procedural memory representations. Alternatively, iconic information from gesture may contribute to declarative memory representations mediated by the hippocampus. To investigate these alternatives, we examined gesture's effects on word learning in patients with hippocampal damage and declarative memory impairment, with intact procedural memory, and in healthy and in brain-damaged comparison groups. Participants learned novel label-object pairings while producing gesture, observing gesture, or observing without gesture. After a delay, recall and object identification were assessed. Unsurprisingly, amnesic patients were unable to recall the labels at test. However, they correctly identified objects at above chance levels, but only if they produced a gesture at encoding. Comparison groups performed well above chance at both recall and object identification regardless of gesture. These findings suggest that gesture production may support word learning by engaging nondeclarative (procedural) memory.

The influence of the hippocampus and declarative memory on word use: Patients with amnesia use less imageable words.

Hippocampal functioning contributes to our ability to generate multifaceted, imagistic event representations. Patients with hippocampal damage produce event narratives that contain fewer details and fewer imagistic features. We hypothesized that impoverished memory representations would influence language at the word level, yielding words lower in imageability and concreteness. We tested this by examining language produced by patients with bilateral hippocampal damage and severe declarative memory impairment, and brain-damaged and healthy comparison groups. Participants described events from the real past, imagined past, imagined present, and imagined future. We analyzed the imageability and concreteness of words used. Patients with amnesia used words that were less imageable than those of comparison groups across time periods, even when accounting for the amount of episodic detail in narratives. Moreover, all participants used words that were relatively more imageable when discussing real past events than other time periods. Taken together, these findings suggest that the memory that we have for an event affects how we talk about that event, and this extends all the way to the individual words that we use.

A technique for continuous measurement of body movement from video.

The movements that we make with our body vary continuously along multiple dimensions. However, many of the tools and techniques presently used for coding and analyzing hand gestures and other body movements yield categorical outcome variables. Focusing on categorical variables as the primary quantitative outcomes may mislead researchers or distort conclusions. Moreover, categorical systems may fail to capture the richness present in movement. Variations in body movement may be informative in multiple dimensions. For example, a single hand gesture has a unique size, height of production, trajectory, speed, and handshape. Slight variations in any of these features may alter how both the speaker and the listener are affected by gesture. In this paper, we describe a new method for measuring and visualizing the physical trajectory of movement using video. This method is generally accessible, requiring only video data and freely available computer software. This method allows researchers to examine features of hand gestures, body movement, and other motion, including size, height, curvature, and speed. We offer a detailed account of how to implement this approach, and we also offer some guidelines for situations where this approach may be fruitful in revealing how the body expresses information. Finally, we provide data from a small study on how speakers alter their hand gestures in response to different characteristics of a stimulus to demonstrate the utility of analyzing continuous dimensions of motion. By creating shared methods, we hope to facilitate communication between researchers from varying methodological traditions.

Hand Gesture and Mathematics Learning: Lessons From an Avatar.

A beneficial effect of gesture on learning has been demonstrated in multiple domains, including mathematics, science, and foreign language vocabulary. However, because gesture is known to co-vary with other non-verbal behaviors, including eye gaze and prosody along with face, lip, and body movements, it is possible the beneficial effect of gesture is instead attributable to these other behaviors. We used a computer-generated animated pedagogical agent to control both verbal and non-verbal behavior. Children viewed lessons on mathematical equivalence in which an avatar either gestured or did not gesture, while eye gaze, head position, and lip movements remained identical across gesture conditions. Children who observed the gesturing avatar learned more, and they solved problems more quickly. Moreover, those children who learned were more likely to transfer and generalize their knowledge. These findings provide converging evidence that gesture facilitates math learning, and they reveal the potential for using technology to study non-verbal behavior in controlled experiments.

Hippocampal declarative memory supports gesture production: Evidence from amnesia.

Spontaneous co-speech hand gestures provide a visuospatial representation of what is being communicated in spoken language. Although it is clear that gestures emerge from representations in memory for what is being communicated (De Ruiter, 1998; Wesp, Hesse, Keutmann, & Wheaton, 2001), the mechanism supporting the relationship between gesture and memory is unknown. Current theories of gesture production posit that action - supported by motor areas of the brain - is key in determining whether gestures are produced. We propose that when and how gestures are produced is determined in part by hippocampally-mediated declarative memory. We examined the speech and gesture of healthy older adults and of memory-impaired patients with hippocampal amnesia during four discourse tasks that required accessing episodes and information from the remote past. Consistent with previous reports of impoverished spoken language in patients with hippocampal amnesia, we predicted that these patients, who have difficulty generating multifaceted declarative memory representations, may in turn have impoverished gesture production. We found that patients gestured less overall relative to healthy comparison participants, and that this was particularly evident in tasks that may rely more heavily on declarative memory. Thus, gestures do not just emerge from the motor representation activated for speaking, but are also sensitive to the representation available in hippocampal declarative memory, suggesting a direct link between memory and gesture production.

Bridging gaps in common ground: Speakers design their gestures for their listeners.

Communication is shaped both by what we are trying to say and by whom we are saying it to. We examined whether and how shared information influences the gestures speakers produce along with their speech. Unlike prior work examining effects of common ground on speech and gesture, we examined a situation in which some speakers have the same amount of mutually shared experience with their listener but the relevance of the information from shared experience is different for listeners in different conditions. Additionally, speakers and listeners in all conditions shared a visual perspective. Speakers and listeners solved a version of the Tower of Hanoi task together. Speakers then solved a second version of the task without the listener present with the manner of disk movement manipulated; the manner was either the same as or different from the version that had been solved with the listener present. Thus, speakers' knowledge of the relevance of shared knowledge was manipulated. We measured the content of speech along with the physical form and content of the accompanying hand gesture. Although speakers did not modulate their spoken language, speakers who knew their listeners had not previously experienced the appropriate manner of completion gestured higher in space, highlighting manner information, but without altering the physical gesture trajectory. Thus, gesture can be sensitive to the knowledge of listeners even when speech is not. Speakers' gestures can play an independent role in reflecting common ground between speakers and listeners, perhaps by simultaneously incorporating both speaker and listener perspectives. (PsycINFO Database Record

Mothers modulate their gesture independently of their speech.

Do speakers alter their gestures independently of speech during communication? We addressed this question by examining how mothers modulate their speech and gestures when communicating about safety with their children. Mothers and their 8- or 10-year-old children viewed and discussed a series of images depicting another child engaged in a variety of physical activities with the goal of deciding on a joint safety rating for each image. When mothers perceived a situation as more unsafe than their child did, they conveyed more information in both speech and gesture. Importantly, as this disparity between mother and child ratings grew, mothers systematically increased their rate of gesturing when communicating dangerous information and decreased their rate of gesturing when communicating non-dangerous information. These findings show that speakers selectively alter their gestures for their listeners, demonstrating that speech and gesture need not be modulated in parallel.

Consolidation and transfer of learning after observing hand gesture.

Children who observe gesture while learning mathematics perform better than children who do not, when tested immediately after training. How does observing gesture influence learning over time? Children (n = 184, ages = 7-10) were instructed with a videotaped lesson on mathematical equivalence and tested immediately after training and 24 hr later. The lesson either included speech and gesture or only speech. Children who saw gesture performed better overall and performance improved after 24 hr. Children who only heard speech did not improve after the delay. The gesture group also showed stronger transfer to different problem types. These findings suggest that gesture enhances learning of abstract concepts and affects how learning is consolidated over time.

Gestures, but not meaningless movements, lighten working memory load when explaining math.

Gesturing is ubiquitous in communication and serves an important function for listeners, who are able to glean meaningful information from the gestures they see. But gesturing also functions for speakers, whose own gestures reduce demands on their working memory. Here we ask whether gesture's beneficial effects on working memory stem from its properties as a rhythmic movement, or as a vehicle for representing meaning. We asked speakers to remember letters while explaining their solutions to math problems and producing varying types of movements. Speakers recalled significantly more letters when producing movements that coordinated with the meaning of the accompanying speech, i.e., when gesturing, than when producing meaningless movements or no movement. The beneficial effects that accrue to speakers when gesturing thus seem to stem not merely from the fact that their hands are moving, but from the fact that their hands are moving in coordination with the content of speech.

Gesturing makes memories that last.

When people are asked to perform actions, they remember those actions better than if they are asked to talk about the same actions. But when people talk, they often gesture with their hands, thus adding an action component to talking. The question we asked in this study was whether producing gesture along with speech makes the information encoded in that speech more memorable than it would have been without gesture. We found that gesturing during encoding led to better recall, even when the amount of speech produced during encoding was controlled. Gesturing during encoding improved recall whether the speaker chose to gesture spontaneously or was instructed to gesture. Thus, gesturing during encoding seems to function like action in facilitating memory.

Embodied communication: speakers' gestures affect listeners' actions.

We explored how speakers and listeners use hand gestures as a source of perceptual-motor information during naturalistic communication. After solving the Tower of Hanoi task either with real objects or on a computer, speakers explained the task to listeners. Speakers' hand gestures, but not their speech, reflected properties of the particular objects and the actions that they had previously used to solve the task. Speakers who solved the problem with real objects used more grasping handshapes and produced more curved trajectories during the explanation. Listeners who observed explanations from speakers who had previously solved the problem with real objects subsequently treated computer objects more like real objects; their mouse trajectories revealed that they lifted the objects in conjunction with moving them sideways, and this behavior was related to the particular gestures that were observed. These findings demonstrate that hand gestures are a reliable source of perceptual-motor information during human communication.

Gesturing gives children new ideas about math.

How does gesturing help children learn? Gesturing might encourage children to extract meaning implicit in their hand movements. If so, children should be sensitive to the particular movements they produce and learn accordingly. Alternatively, all that may matter is that children move their hands. If so, they should learn regardless of which movements they produce. To investigate these alternatives, we manipulated gesturing during a math lesson. We found that children required to produce correct gestures learned more than children required to produce partially correct gestures, who learned more than children required to produce no gestures. This effect was mediated by whether children took information conveyed solely in their gestures and added it to their speech. The findings suggest that body movements are involved not only in processing old ideas, but also in creating new ones. We may be able to lay foundations for new knowledge simply by telling learners how to move their hands.

Gesturing makes learning last.

The gestures children spontaneously produce when explaining a task predict whether they will subsequently learn that task. Why? Gesture might simply reflect a child's readiness to learn a particular task. Alternatively, gesture might itself play a role in learning the task. To investigate these alternatives, we experimentally manipulated children's gesture during instruction in a new mathematical concept. We found that requiring children to gesture while learning the new concept helped them retain the knowledge they had gained during instruction. In contrast, requiring children to speak, but not gesture, while learning the concept had no effect on solidifying learning. Gesturing can thus play a causal role in learning, perhaps by giving learners an alternative, embodied way of representing new ideas. We may be able to improve children's learning just by encouraging them to move their hands.