A sensorimotor perspective on numerical cognition.

Numbers are present in every part of modern society and the human capacity to use numbers is unparalleled in other species. Understanding the mental and neural representations supporting this capacity is of central interest to cognitive psychology, neuroscience, and education. Embodied numerical cognition theory suggests that beyond the seemingly abstract symbols used to refer to numbers, their underlying meaning is deeply grounded in sensorimotor experiences, and that our specific understanding of numerical information is shaped by actions related to our fingers, egocentric space, and experiences with magnitudes in everyday life. We propose a sensorimotor perspective on numerical cognition in which number comprehension and numerical proficiency emerge from grounding three distinct numerical core concepts: magnitude, ordinality, and cardinality.

The Force of Numbers: Investigating Manual Signatures of Embodied Number Processing.

The study has two objectives: (1) to introduce grip force recording as a new technique for studying embodied numerical processing; and (2) to demonstrate how three competing accounts of numerical magnitude representation can be tested by using this new technique: the Mental Number Line (MNL), A Theory of Magnitude (ATOM) and Embodied Cognition (finger counting-based) account. While 26 healthy adults processed visually presented single digits in a go/no-go n-back paradigm, their passive holding forces for two small sensors were recorded in both hands. Spontaneous and unconscious grip force changes related to number magnitude occurred in the left hand already 100-140 ms after stimulus presentation and continued systematically. Our results support a two-step model of number processing where an initial stage is related to the automatic activation of all stimulus properties whereas a later stage consists of deeper conscious processing of the stimulus. This interpretation generalizes previous work with linguistic stimuli and elaborates the timeline of embodied cognition. We hope that the use of grip force recording will advance the field of numerical cognition research.

Stimulating numbers: signatures of finger counting in numerosity processing.

Finger counting is one of the first steps in the development of mature number concepts. With a one-to-one correspondence of fingers to numbers in Western finger counting, fingers hold two numerical meanings: one is based on the number of fingers raised and the second is based on their ordinal position within the habitual finger counting sequence. This study investigated how these two numerical meanings of fingers are intertwined with numerical cognition in adults. Participants received tactile stimulation on their fingertips of one hand and named either the number of fingers stimulated (2, 3, or 4 fingers; Experiment 1) or the number of stimulations on one fingertip (2, 3, or 4 stimulations; Experiment 2). Responses were faster and more accurate when the set of stimulated fingers corresponded to finger counting habits (Experiment 1) and when the number of stimulations matched the ordinal position of the stimulated finger (Experiment 2). These results show that tactile numerosity perception is affected by individual finger counting habits and that those habits give numerical meaning to single fingers.

Incidental Counting: Speeded Number Naming Through Finger Movements.

The first steps in numerical cognition are usually done in conjunction with fingers. Following the assumption that abstract concepts stay associated with the sensory-motor information that was present during their acquisition and consolidation, mental number representations should always be associated with the respective finger counting components. We tested whether finger movements that imply finger counting actually prime the corresponding number concepts in adults. All participants counted number 1 with their thumb and incremented sequentially to number 5 with their pinky. In the experiment, participants sequentially and repeatedly pressed five buttons from thumb to pinky. Each button press triggered the visual presentation of a random number between 1 and 5 that had to be named aloud, resulting in 20% counting-congruent and 80% counting-incongruent finger-number mappings. Average naming latencies were significantly shorter for congruent than incongruent finger-number combinations. Furthermore, there was a distance effect where primes partly co-activated numerically close target numbers and with decreasing priming for more distant prime-target pairs. Overall, these results provide further evidence that number representations are strongly associated with finger counting experience, making fingers an effective tool for number comprehension.

Two Attributes of Number Meaning.

Many studies demonstrated interactions between number processing and either spatial codes (effects of spatial-numerical associations) or visual size-related codes (size-congruity effect). However, the interrelatedness of these two number couplings is still unclear. The present study examines the simultaneous occurrence of space- and size-numerical congruency effects and their interactions both within and across trials. In a magnitude judgment task physically small or large digits were presented left or right from screen center. The reaction times analysis revealed that space- and size-congruency effects coexisted in parallel and combined additively. Moreover, a selective sequential modulation of the two congruency effects was found. The size-congruency effect was reduced after size incongruent trials. The space-congruency effect, however, was only affected by the previous space congruency. The observed independence of spatial-numerical and within-magnitude associations is interpreted as evidence that the two couplings reflect different attributes of numerical meaning possibly related to ordinality and cardinality.

Finger posing primes number comprehension.

Canonical finger postures, as used in counting, activate number knowledge, but the exact mechanism for this priming effect is unclear. Here we dissociated effects of visual versus motor priming of number concepts. In Experiment 1, participants were exposed either to pictures of canonical finger postures (visual priming) or actively produced the same finger postures (motor priming) and then used foot responses to rapidly classify auditory numbers (targets) as smaller or larger than 5. Classification times revealed that manually adopted but not visually perceived postures primed magnitude classifications. Experiment 2 obtained motor priming of number processing through finger postures also with vocal responses. Priming only occurred through canonical and not through non-canonical finger postures. Together, these results provide clear evidence for motor priming of number knowledge. Relative contributions of vision and action for embodied numerical cognition and the importance of canonicity of postures are discussed.

Interaction between numbers and size during visual search.

The current study investigates an interaction between numbers and physical size (i.e. size congruity) in visual search. In three experiments, participants had to detect a physically large (or small) target item among physically small (or large) distractors in a search task comprising single-digit numbers. The relative numerical size of the digits was varied, such that the target item was either among the numerically large or small numbers in the search display and the relation between numerical and physical size was either congruent or incongruent. Perceptual differences of the stimuli were controlled by a condition in which participants had to search for a differently coloured target item with the same physical size and by the usage of LCD-style numbers that were matched in visual similarity by shape transformations. The results of all three experiments consistently revealed that detecting a physically large target item is significantly faster when the numerical size of the target item is large as well (congruent), compared to when it is small (incongruent). This novel finding of a size congruity effect in visual search demonstrates an interaction between numerical and physical size in an experimental setting beyond typically used binary comparison tasks, and provides important new evidence for the notion of shared cognitive codes for numbers and sensorimotor magnitudes. Theoretical consequences for recent models on attention, magnitude representation and their interactions are discussed.

A simple technique to study embodied language processes: the grip force sensor.

Research in cognitive neuroscience has shown that brain structures serving perceptual, emotional, and motor processes are also recruited during the understanding of language when it refers to emotion, perception, and action. However, the exact linguistic and extralinguistic conditions under which such language-induced activity in modality-specific cortex is triggered are not yet well understood. The purpose of this study is to introduce a simple experimental technique that allows for the online measure of language-induced activity in motor structures of the brain. This technique consists in the use of a grip force sensor that captures subtle grip force variations while participants listen to words and sentences. Since grip force reflects activity in motor brain structures, the continuous monitoring of force fluctuations provides a fine-grained estimation of motor activity across time. In other terms, this method allows for both localization of the source of language-induced activity to motor brain structures and high temporal resolution of the recorded data. To facilitate comparison of the data to be collected with this tool, we present two experiments that describe in detail the technical setup, the nature of the recorded data, and the analyses (including justification about the data filtering and artifact rejection) that we applied. We also discuss how the tool could be used in other domains of behavioral research.

Spatial interferences in mental arithmetic: evidence from the motion-arithmetic compatibility effect.

Recent research on spatial number representations suggests that the number space is not necessarily horizontally organized and might also be affected by acquired associations between magnitude and sensory experiences in vertical space. Evidence for this claim is, however, controversial. The present study now aims to compare vertical and horizontal spatial associations in mental arithmetic. In Experiment 1, participants solved addition and subtraction problems and indicated the result verbally while moving their outstretched right arm continuously left-, right-, up-, or downwards. The analysis of the problem-solving performances revealed a motion-arithmetic compatibility effect for spatial actions along both the horizontal and the vertical axes. Performances in additions was impaired while making downward compared to upward movements as well as when moving left compared to right and vice versa in subtractions. In Experiment 2, instead of being instructed to perform active body movements, participants calculated while the problems moved in one of the four relative directions on the screen. For visual motions, only the motion-arithmetic compatibility effect for the vertical dimension could be replicated. Taken together, our findings provide first evidence for an impact of spatial processing on mental arithmetic. Moreover, the stronger effect of the vertical dimension supports the idea that mental calculations operate on representations of numerical magnitude that are grounded in a vertically organized mental number space.

Different brains process numbers differently: structural bases of individual differences in spatial and nonspatial number representations.

A dominant hypothesis on how the brain processes numerical size proposes a spatial representation of numbers as positions on a "mental number line." An alternative hypothesis considers numbers as elements of a generalized representation of sensorimotor-related magnitude, which is not obligatorily spatial. Here we show that individuals' relative use of spatial and nonspatial representations has a cerebral counterpart in the structural organization of the posterior parietal cortex. Interindividual variability in the linkage between numbers and spatial responses (faster left responses to small numbers and right responses to large numbers; spatial-numerical association of response codes effect) correlated with variations in gray matter volume around the right precuneus. Conversely, differences in the disposition to link numbers to force production (faster soft responses to small numbers and hard responses to large numbers) were related to gray matter volume in the left angular gyrus. This finding suggests that numerical cognition relies on multiple mental representations of analogue magnitude using different neural implementations that are linked to individual traits.

Expyriment: a Python library for cognitive and neuroscientific experiments.

Expyriment is an open-source and platform-independent lightweight Python library for designing and conducting timing-critical behavioral and neuroimaging experiments. The major goal is to provide a well-structured Python library for script-based experiment development, with a high priority being the readability of the resulting program code. Expyriment has been tested extensively under Linux and Windows and is an all-in-one solution, as it handles stimulus presentation, the recording of input/output events, communication with other devices, and the collection and preprocessing of data. Furthermore, it offers a hierarchical design structure, which allows for an intuitive transition from the experimental design to a running program. It is therefore also suited for students, as well as for experimental psychologists and neuroscientists with little programming experience.

Embodied grounding of memory: toward the effects of motor execution on memory consolidation.

Behavioural and neuroscientific research has provided evidence for a strong functional link between the neural motor system and lexical-semantic processing of action-related language. It remains unclear, however, whether the impact of motor actions is restricted to online language comprehension or whether sensorimotor codes are also important in the formation and consolidation of persisting memory representations of the word's referents. The current study now demonstrates that recognition performance for action words is modulated by motor actions performed during the retention interval. Specifically, participants were required to learn words denoting objects that were associated with either a pressing or a twisting action (e.g., piano, screwdriver) and words that were not associated to actions. During a 6-8-minute retention phase, participants performed an intervening task that required the execution of pressing or twisting responses. A subsequent recognition task revealed a better memory for words that denoted objects for which the functional use was congruent with the action performed during the retention interval (e.g., pepper mill-twisting action, doorbell-pressing action) than for words that denoted objects for which the functional use was incongruent. In further experiments, we were able to generalize this effect of selective memory enhancement of words by performing congruent motor actions to an implicit perceptual (Experiment 2) and implicit semantic memory test (Experiment 3). Our findings suggest that a reactivation of motor codes affects the process of memory consolidation and emphasizes therefore the important role of sensorimotor codes in establishing enduring semantic representations.

A feeling for numbers: shared metric for symbolic and tactile numerosities.

Evidence for an approximate analog system of numbers has been provided by the finding that the comparison of two numerals takes longer and is more error-prone if the semantic distance between the numbers becomes smaller (so-called numerical distance effect). Recent embodied theories suggest that analog number representations are based on previous sensory experiences and constitute therefore a common magnitude metric shared by multiple domains. Here we demonstrate the existence of a cross-modal semantic distance effect between symbolic and tactile numerosities. Participants received tactile stimulations of different amounts of fingers while reading Arabic digits and indicated verbally whether the amount of stimulated fingers was different from the simultaneously presented digit or not. The larger the semantic distance was between the two numerosities, the faster and more accurate participants made their judgments. This cross-modal numerosity distance effect suggests a direct connection between tactile sensations and the concept of numerical magnitude. A second experiment replicated the interaction between symbolic and tactile numerosities and showed that this effect is not modulated by the participants' finger counting habits. Taken together, our data provide novel evidence for a shared metric for symbolic and tactile numerosities as an instance of an embodied representation of numbers.

Action-effect binding by observational learning.

The acquisition of bidirectional action-effect associations plays a central role in the ability to intentionally control actions. Humans learn about actions not only through active experience, but also through observing the actions of others. In Experiment 1, we examined whether action-effect associations can be acquired by observational learning. To this end, participants observed how a model repeatedly pressed two buttons during an observation phase. Each of the buttonpresses led to a specific tone (action effect). In a subsequent test phase, the tones served as target stimuli to which the participants had to respond with buttonpresses. Reaction times were shorter if the stimulus-response mapping in the test phase was compatible with the action-effect association in the observation phase. Experiment 2 excluded the possibility that the impact of perceived action effects on own actions was driven merely by an association of spatial features with the particular tones. Furthermore, we demonstrated that the presence of an agent is necessary to acquire novel action-effect associations through observation. Altogether, the study provides evidence for the claim that bidirectional action-effect associations can be acquired by observational learning. Our findings are discussed in the context of the idea that the acquisition of action-effect associations through observation is an important cognitive mechanism subserving the human ability for social learning.

Grasping the other's attention: the role of animacy in action cueing of joint attention.

The current experiment investigates the role of animacy on grasp-cueing effects as investigated in joint attention research. In a simple detection task participants responded to the colour change of one of two objects of identical size. Before the target onset, we presented a cueing stimulus consisting of either two human hands with a small and a large grip aperture (animate condition) or two comparable U-shaped figures with small and large aperture (inanimate condition). Depending on the size of the objects and the arrangement of the apertures (i.e., large aperture to the left and small aperture to the right or vice versa), either the left or right object matched the grasping hand or U-shapes. Our data show that biological grasping actions modulate the observer's attention whereas the perception of inanimate stimuli does not result in a comparable cueing effect. This strong impact of animacy on attentional priming suggests that grasp cueing represents a marker of a joint attention mechanism that involves spontaneous simulation of the observed motor behaviour.

Context effects on the processing of action-relevant object features.

In 4 experiments, we investigated the effects of object affordance in reach-to-grasp actions. Participants indicated whether a depicted small or large object was natural or manmade by means of different object-grasping responses (i.e., with a power or a precision grip). We observed that the size of the depicted object affected the grasping kinematics (grip aperture) and the reach-onset times of compatible and incompatible actions. Additional experiments showed that the effect of perceived object size on motor response was modulated by contextual action information and the observation of others' actions with the object. Thus, beyond the observation of object affordance effects in natural grasping actions, this study suggests that the coupling between object perception and action is not static and obligatory. Behavioral effects of action-relevant object features seem rather to depend on contextual action information.

Effects of intentional motor actions on embodied language processing.

Embodied theories of language processing suggest that this motor simulation is an automatic and necessary component of meaning representation. If this is the case, then language and action systems should be mutually dependent (i.e., motor activity should selectively modulate processing of words with an action-semantic component). In this paper, we investigate in two experiments whether evidence for mutual dependence can be found using a motor priming paradigm. Specifically, participants performed either an intentional or a passive motor task while processing words denoting manipulable and nonmanipulable objects. The performance rates (Experiment 1) and response latencies (Experiment 2) in a lexical-decision task reveal that participants performing an intentional action were positively affected in the processing of words denoting manipulable objects as compared to nonmanipulable objects. This was not the case if participants performed a secondary passive motor action (Experiment 1) or did not perform a secondary motor task (Experiment 2). The results go beyond previous research showing that language processes involve motor systems to demonstrate that the execution of motor actions has a selective effect on the semantic processing of words. We suggest that intentional actions activate specific parts of the neural motor system, which are also engaged for lexical-semantic processing of action-related words and discuss the beneficial versus inhibitory nature of this relationship. The results provide new insights into the embodiment of language and the bidirectionality of effects between language and action processing.

Context effects in embodied lexical-semantic processing.

The embodied view of language comprehension proposes that the meaning of words is grounded in perception and action rather than represented in abstract amodal symbols. Support for embodied theories of language processing comes from behavioral studies showing that understanding a sentence about an action can modulate congruent and incongruent physical responses, suggesting motor involvement during comprehension of sentences referring to bodily movement. Additionally, several neuroimaging studies have provided evidence that comprehending single words denoting manipulable objects elicits specific responses in the neural motor system. An interesting question that remains is whether action semantic knowledge is directly activated as motor simulations in the brain, or rather modulated by the semantic context in which action words are encountered. In the current paper we investigated the nature of conceptual representations using a go/no-go lexical decision task. Specifically, target words were either presented in a semantic context that emphasized dominant action features (features related to the functional use of an object) or non-dominant action features. The response latencies in a lexical decision task reveal that participants were faster to respond to words denoting objects for which the functional use was congruent with the prepared movement. This facilitation effect, however, was only apparent when the semantic context emphasized corresponding motor properties. These findings suggest that conceptual processing is a context-dependent process that incorporates motor-related knowledge in a flexible manner.

The function of words: distinct neural correlates for words denoting differently manipulable objects.

Recent research indicates that language processing relies on brain areas dedicated to perception and action. For example, processing words denoting manipulable objects has been shown to activate a fronto-parietal network involved in actual tool use. This is suggested to reflect the knowledge the subject has about how objects are moved and used. However, information about how to use an object may be much more central to the conceptual representation of an object than information about how to move an object. Therefore, there may be much more fine-grained distinctions between objects on the neural level, especially related to the usability of manipulable objects. In the current study, we investigated whether a distinction can be made between words denoting (1) objects that can be picked up to move (e.g., volumetrically manipulable objects: bookend, clock) and (2) objects that must be picked up to use (e.g., functionally manipulable objects: cup, pen). The results show that functionally manipulable words elicit greater levels of activation in the fronto-parietal sensorimotor areas than volumetrically manipulable words. This suggests that indeed a distinction can be made between different types of manipulable objects. Specifically, how an object is used functionally rather than whether an object can be displaced with the hand is reflected in semantic representations in the brain.

Motor simulation in verbal knowledge acquisition.

Recent research highlights the importance of motor processes for a wide range of cognitive functions such as object perception and language comprehension. It is unclear, however, whether the involvement of the motor system goes beyond the processing of information that is gathered through active action experiences and affects also the representation of knowledge acquired through verbal learning. We tested this prediction by varying the presence of motor interference (i.e., squeezing a ball vs. oddball detection task) while participants verbally acquired functional object knowledge and examined the effects on a subsequent object detection task. Results revealed that learning of functional object knowledge was only impaired when participants performed an effector-specific motor task while training. The present finding of an effector-specific motor interference effect on object learning demonstrates the crucial role of the motor system in the acquisition of novel object knowledge and provides support for an embodied account to perception and cognition.