High-Pitched Sound is Open and Low-Pitched Sound is Closed: Representing the Spatial Meaning of Pitch Height.

Research shows that high- and low-pitch sounds can be associated with various meanings. For example, high-pitch sounds are associated with small concepts, whereas low-pitch sounds are associated with large concepts. This study presents three experiments revealing that high-pitch sounds are also associated with open concepts and opening hand actions, while low-pitch sounds are associated with closed concepts and closing hand actions. In Experiment 1, this sound-meaning correspondence effect was shown using the two-alternative forced-choice task, while Experiments 2 and 3 used reaction time tasks to show this interaction. In Experiment 2, high-pitch vocalizations were found to facilitate opening hand gestures, and low-pitch vocalizations were found to facilitate closing hand gestures, when performed simultaneously. In Experiment 3, high-pitched vocalizations were produced particularly rapidly when the visual target stimulus presented an open object, and low-pitched vocalizations were produced particularly rapidly when the target presented a closed object. These findings are discussed concerning the meaning of intonational cues. They are suggested to be based on cross-modally representing conceptual spatial knowledge in sensory, motor, and affective systems. Additionally, this pitch-opening effect might share cognitive processes with other pitch-meaning effects.

Is Front associated with Above and Back with Below? Association between Allocentric Representations of Spatial Dimensions.

Previous research has revealed congruency effects between different spatial dimensions such as right and up. In the audiovisual context, high-pitched sounds are associated with the spatial dimensions of up/above and front, while low-pitched sounds are associated with the spatial dimensions of down/below and back. This opens the question of whether there could also be a spatial association between above and front and/or below and back. Participants were presented with a high- or low-pitch stimulus at the time of the onset of the visual stimulus. In one block, participants responded according to the above/below location of the visual target stimulus if the target appeared in front of the reference object, and in the other block, they performed these above/below responses if the target appeared at the back of the reference. In general, reaction times revealed an advantage in processing the target location in the front-above and back-below locations. The front-above/back-below effect was more robust concerning the back-below component of the effect, and significantly larger in reaction times that were slower rather than faster than the median value of a participant. However, the pitch did not robustly influence responding to front/back or above/below locations. We propose that this effect might be based on the conceptual association between different spatial dimensions.

Sound-action symbolism in relation to precision manipulation and whole-hand grasp usage.

It has been suggested that actions can provide a fruitful conceptual context for sound symbolism phenomena, and that tight interaction between manual and articulatory processes might cause that hand actions, in particular, are sound-symbolically associated with specific speech sounds. Experiment 1 investigated whether novel words, built from speech sounds that have been previously linked to precision or power grasp responses, are implicitly associated with perceived actions that present precision manipulation or whole-hand grasp tool-use or the corresponding utilisation pantomimes. In the two-alternative forced-choice task, the participants were more likely to match novel words to tool-use actions and corresponding pantomimes that were sound-symbolically congruent with the words. Experiment 2 showed that the same or even larger sound-action symbolism effect can be observed when the pantomimes present unfamiliar utilisation actions. Based on this we propose that the sound-action symbolism might originate from the same sensorimotor mechanisms that process the meaning of iconic gestural signs. The study presents a novel sound-action phenomenon and supports the view that hand-mouth interaction might manifest itself by associating specific speech sounds with grasp-related utilisations.

Front Is High and Back Is Low: Sound-Space Iconicity in Finnish.

Previous investigations have shown various interactions between spatial concepts and speech sounds. For instance, the front-high vowel [i] is associated with the concept of forward, and the back-high vowel [o] is associated with the concept of backward. Three experiments investigated whether the concepts of forward/front and backward/back are associated with high- and low-pitched vocalizations, respectively, in Finnish. In Experiments 1 and 2, the participants associated the high-pitched vocalization with the forward-directed movement and the low-pitched vocalizations with the backward-directed movement. In Experiment 3, the same effect was observed in relation to the concepts of front of and back of. We propose that these observations present a novel sound-space symbolism phenomenon in which spatial concepts of forward/front and backward/back are iconically associated with high- and low-pitched speech sounds. This observation is discussed in relation to the grounding of semantic knowledge of these spatial concepts in the movements of articulators such as relative front/back-directed movements of the tongue.

Auditory-perceptual evaluation with visual analogue scale: feasibility and preliminary evidence of ultrasound visual feedback treatment of Finnish [r].

Accumulating evidence suggests that ultrasound visual feedback increases the treatment efficacy for persistent speech sound errors. However, the available evidence is mostly from English. This is a feasibility study of ultrasound visual feedback for treating distortion of Finnish [r]. We developed a web-based application for auditory-perceptual judgement. We investigated the impact of listener's experience on perceptual judgement and the intra-rater reliability of listeners. Four boys (10-11 years) with distortion of [r], otherwise typical development, partook in eight ultrasound treatment sessions. In total, 117 [r] samples collected at pre- and post-intervention were judged with visual analogue scale (VAS) by two listener groups: five speech and language therapists (SLTs) and six SLT students. We constructed a linear mixed-effects model with fixed effects for time and listener group and several random effects. Our findings indicate that measurement time had a significant main effect on judgement results, χ2 = 78.82, p < 0.001. Effect of listener group was non-significant, but a significant main effect of interaction of group × time, χ2 = 6.33, p < 0.012 was observed. We further explored the effect of group with nested models, and results revealed a non-significant effect of group. The average intra-rater correlation of the 11 listeners was 0.83 for the pre-intervention samples and 0.92 for post-intervention showing a good or excellent degree of agreement. Finnish [r] sound can be evaluated with VAS and ultrasound visual feedback is a feasible and promising method in treatment for distortion of [r], and its efficacy should be further assessed.

Interaction between grasping and articulation: How vowel and consonant pronunciation influences precision and power grip responses.

Grasping and mouth movements have been proposed to be integrated anatomically, functionally and evolutionarily. In line with this, we have shown that there is a systematic interaction between particular speech units and grip performance. For example, when the task requires pronouncing a speech unit simultaneously with grasp response, the speech units [i] and [t] are associated with relatively rapid and accurate precision grip responses, while [ɑ] and [k] are associated with power grip responses. This study is aimed at complementing the picture about which vowels and consonants are associated with these grasp types. The study validated our view that the high-front vowels and the alveolar consonants are associated with precision grip responses, while low and high-back vowels as well as velar consonants or those whose articulation involves the lowering of the tongue body are associated with power grip responses. This paper also proposes that one reason why small/large concepts are associated with specific speech sounds in the sound-magnitude symbolism is because articulation of these sounds is programmed within the overlapping mechanisms of precision or power grasping.

Sound-Action Symbolism.

Recent evidence has shown linkages between actions and segmental elements of speech. For instance, close-front vowels are sound symbolically associated with the precision grip, and front vowels are associated with forward-directed limb movements. The current review article presents a variety of such sound-action effects and proposes that they compose a category of sound symbolism that is based on grounding a conceptual knowledge of a referent in articulatory and manual action representations. In addition, the article proposes that even some widely known sound symbolism phenomena such as the sound-magnitude symbolism can be partially based on similar sensorimotor grounding. It is also discussed that meaning of suprasegmental speech elements in many instances is similarly grounded in body actions. Sound symbolism, prosody, and body gestures might originate from the same embodied mechanisms that enable a vivid and iconic expression of a meaning of a referent to the recipient.

Dialectal variation of duration patterns in Finnmark North Sámi quantity.

Ternary length contrast is a rare phonological feature, investigated here both in terms of its realization and possible undergoing changes. In North Sámi, a phonetically under-documented and endangered Fenno-Ugric language spoken by indigenous people in Northern Europe, the ternary quantity contrast is assumed to be signalled by a progressive lengthening of a consonant and a compensatory shortening of the previous vowel. This study evaluates this assumption and compares the realization of the length contrasts in two dialects, the Western and Eastern Finnmark North Sámi. The results show that while the contrast between the short and the two longer quantities is robustly signaled regardless of the dialect, the durational differences between the two longer quantities are maintained only in the Eastern dialect. On the other hand, a vowel quantity contrast independent of the quantity of the following consonant is present in the Western but not in the Eastern dialect. Further, comparing the phonetic realization of the ternary quantity contrast for speakers of different ages presents evidence of a language change: the results indicate an ongoing neutralization of the ternary contrast in younger speakers, which points to a possible disappearance of this rare typological feature in Finnmark North Sámi.

The Sound of Grasp Affordances: Influence of Grasp-Related Size of Categorized Objects on Vocalization.

Previous research shows that simultaneously executed grasp and vocalization responses are faster when the precision grip is performed with the vowel [i] and the power grip is performed with the vowel [ɑ]. Research also shows that observing an object that is graspable with a precision or power grip can activate the grip congruent with the object. Given the connection between vowel articulation and grasping, this study explores whether grasp-related size of observed objects can influence not only grasp responses but also vowel pronunciation. The participants had to categorize small and large objects into natural and manufactured categories by pronouncing the vowel [i] or [ɑ]. As predicted, [i] was produced faster when the object's grasp-related size was congruent with the precision grip while [ɑ] was produced faster when the size was congruent with the power grip (Experiment 1). The effect was not, however, observed when the participants were presented with large objects that are not typically grasped by the power grip (Experiment 2). This study demonstrates that vowel production is systematically influenced by grasp-related size of a viewed object, supporting the account that sensory-motor processes related to grasp planning and representing grasp-related properties of viewed objects interact with articulation processes. The paper discusses these findings in the context of size-sound symbolism, suggesting that mechanisms that transform size-grasp affordances into corresponding grasp- and articulation-related motor programs might provide a neural basis for size-sound phenomena that links small objects with closed-front vowels and large objects with open-back vowels.

Connecting directional limb movements to vowel fronting and backing.

It has been shown recently that when participants are required to pronounce a vowel at the same time with the hand movement, the vocal and manual responses are facilitated when a front vowel is produced with forward-directed hand movements and a back vowel is produced with backward-directed hand movements. This finding suggests a coupling between spatial programing of articulatory tongue movements and hand movements. The present study revealed that the same effect can be also observed in relation to directional leg movements. The study suggests that the effect operates within the common directional processes of movement planning including at least tongue, hand and leg movements, and these processes might contribute sound-to-meaning mappings to the semantic concepts of 'forward' and 'backward'.

Reaching and grasping with the tongue: Shared motor planning between hand actions and articulatory gestures.

Research has shown connections between articulatory mouth actions and manual actions. This study investigates whether forward-backward hand movements could be associated with vowel production processes that programme tongue fronting/backing, lip rounding/spreading (Experiment 1), and/or consonant production processes that programme tongue tip and tongue dorsum actions (Experiment 2). The participants had to perform either forward or backward hand movement and simultaneously pronounce different vowels or consonants. The results revealed a response benefit, measured in vocal and manual reaction times, when the responses consisted of front vowels and forward hand movements. Conversely, back vowels were associated with backward hand movements. Articulation of rounded versus unrounded vowels or coronal versus dorsal consonants did not produce the effect. In contrast, when the manual responses of forward-backward hand movements were replaced by precision and power grip responses, the coronal consonants [t] and [r] were associated with the precision grip, whereas the dorsal consonant [k] was associated with the power grip. We propose that the movements of the tongue body, operating mainly for vowel production, share the directional action planning processes with the hand movements. Conversely, the tongue articulators related to tongue tip and dorsum movements, operating mainly for consonant production, share the action planning processes with the precision and power grip, respectively.

The influence of fundamental frequency on perceived duration in spectrally comparable sounds.

The perceived duration of a sound is affected by its fundamental frequency and intensity: higher sounds are judged to be longer, as are sounds with greater intensity. Since increasing intensity lengthens the perceived duration of the auditory object, and increasing the fundamental frequency increases the sound's perceived loudness (up to ca. 3 kHz), frequency modulation of duration could be potentially explained by a confounding effect where the primary cause of the modulation would be variations in intensity. Here, a series of experiments are described that were designed to disentangle the contributions of fundamental frequency, intensity, and duration to perceived loudness and duration. In two forced-choice tasks, participants judged duration and intensity differences between two sounds varying simultaneously in intensity, fundamental frequency, fundamental frequency gliding range, and duration. The results suggest that fundamental frequency and intensity each have an impact on duration judgments, while frequency gliding range did not influence the present results. We also demonstrate that the modulation of perceived duration by sound fundamental frequency cannot be fully explained by the confounding relationship between frequency and intensity.

Mismatch negativity (MMN) to speech sounds is modulated systematically by manual grip execution.

Manual actions and speech are connected: for example, grip execution can influence simultaneous vocalizations and vice versa. Our previous studies show that the consonant [k] is associated with the power grip and the consonant [t] with the precision grip. Here we studied whether the interaction between speech sounds and grips could operate already at a pre-attentive stage of auditory processing, reflected by the mismatch-negativity (MMN) component of the event-related potential (ERP). Participants executed power and precision grips according to visual cues while listening to syllable sequences consisting of [ke] and [te] utterances. The grips modulated the MMN amplitudes to these syllables in a systematic manner so that when the deviant was [ke], the MMN response was larger with a precision grip than with a power grip. There was a converse trend when the deviant was [te]. These results suggest that manual gestures and speech can interact already at a pre-attentive processing level of auditory perception, and show, for the first time that manual actions can systematically modulate the MMN.

Connections of Grasping and Horizontal Hand Movements with Articulation in Czech Speakers.

We have recently shown in Finnish speakers that articulation of certain vowels and consonants has a systematic influence on simultaneous grasp actions as well as on forward and backward hand movements. Here we studied whether these effects generalize to another language, namely Czech. We reasoned that if the results generalized to another language environment, it would suggest that the effects arise through other processes than language-dependent semantic associations. Rather, the effects would be likely to arise through language-independent interactions between processes that plan articulatory gestures and hand movements. Participants were presented with visual stimuli specifying articulations to be uttered (e.g., A or I), and they were required to produce a manual response concurrently with the articulation. In Experiment 1 they responded with a precision or a power grip, whereas in Experiment 2 they responded with a forward or a backward hand movement. The grip congruency effect was fully replicated: the consonant [k] and the vowel [α] were associated with power grip responses, while the consonant [t] and the vowel [i] were associated with precision grip responses. The forward/backward congruency effect was replicated with vowels [α], [o], which were associated with backward movement and with [i], which was associated with forward movement, but not with consonants [k] and [t]. These findings suggest that the congruency effects mostly reflect interaction between processes that plan articulatory gestures and hand movements with an exception that the forward/backward congruency effect might only work with vowel articulation.

Musical Sophistication and the Effect of Complexity on Auditory Discrimination in Finnish Speakers.

Musical experiences and native language are both known to affect auditory processing. The present work aims to disentangle the influences of native language phonology and musicality on behavioral and subcortical sound feature processing in a population of musically diverse Finnish speakers as well as to investigate the specificity of enhancement from musical training. Finnish speakers are highly sensitive to duration cues since in Finnish, vowel and consonant duration determine word meaning. Using a correlational approach with a set of behavioral sound feature discrimination tasks, brainstem recordings, and a musical sophistication questionnaire, we find no evidence for an association between musical sophistication and more precise duration processing in Finnish speakers either in the auditory brainstem response or in behavioral tasks, but they do show an enhanced pitch discrimination compared to Finnish speakers with less musical experience and show greater duration modulation in a complex task. These results are consistent with a ceiling effect set for certain sound features which corresponds to the phonology of the native language, leaving an opportunity for music experience-based enhancement of sound features not explicitly encoded in the language (such as pitch, which is not explicitly encoded in Finnish). Finally, the pattern of duration modulation in more musically sophisticated Finnish speakers suggests integrated feature processing for greater efficiency in a real world musical situation. These results have implications for research into the specificity of plasticity in the auditory system as well as to the effects of interaction of specific language features with musical experiences.

Preparation and execution of teeth clenching and foot muscle contraction influence on corticospinal hand-muscle excitability.

Contraction of a muscle modulates not only the corticospinal excitability (CSE) of the contracting muscle but also that of different muscles. We investigated to what extent the CSE of a hand muscle is modulated during preparation and execution of teeth clenching and ipsilateral foot dorsiflexion either separately or in combination. Hand-muscle CSE was estimated based on motor evoked potentials (MEPs) elicited by transcranial magnetic stimulation (TMS) and recorded from the first dorsal interosseous (FDI) muscle. We found higher excitability during both preparation and execution of all the motor tasks than during mere observation of a fixation cross. As expected, the excitability was greater during the execution phase than the preparation one. Furthermore, both execution and preparation of combined motor tasks led to higher excitability than individual tasks. These results extend our current understanding of the neural interactions underlying simultaneous contraction of muscles in different body parts.

Systematic Influence of Perceived Grasp Shape on Speech Production.

Previous research has shown that precision and power grip performance is consistently influenced by simultaneous articulation. For example, power grip responses are performed relatively fast with the open-back vowel [a], whereas precision grip responses are performed relatively fast with the close-front vowel [i]. In the present study, the participants were presented with a picture of a hand shaped to the precision or power grip. They were required to pronounce speech sounds according to the front/above perspective of the hand. The results showed that not only the grip performance is affected by simultaneously pronouncing the speech sound but also the production of speech sound can be affected by viewing an image of a grip. The precision grip stimulus triggered relatively rapid production of the front-close vowel [i]. In contrast, the effect related to the power grip stimulus was mostly linked to the vertical dimension of the pronounced vowel since this stimulus triggered relatively rapid production of the back-open vowel [a] and back-mid-open vowel [o] while production of the back-close vowel [u] was not influenced by it. The fact that production of the dorsal consonant [k] or coronal consonant [t] were not influenced by these stimuli suggests that the effect was not associated with a relative front-back tongue shape of the articulation in the absence of changes in any vertical articulatory components. These findings provide evidence for an intimate interaction between certain articulatory gestures and grip types, suggesting that an overlapping visuomotor network operates for planning articulatory gestures and grasp actions.

Interaction in planning vocalizations and grasping.

Previous studies have shown a congruency effect between manual grasping and syllable articulation. For instance, a power grip is associated with syllables whose articulation involves the tongue body and/or large mouth aperture ([kɑ]) whereas a precision grip is associated with articulations that involve the tongue tip and/or small mouth aperture ([ti]). Previously, this effect has been observed in manual reaction times. The primary aim of the current study was to investigate whether this congruency effect also takes place in vocal responses and to investigate involvement of action selection processes in the effect. The congruency effect was found in vocal and manual responses regardless of whether or not the syllable or grip was known a priori, suggesting that the effect operates with minimal or absent action selection processes. In addition, the effect was observed in vocal responses even when the grip was only prepared but not performed, suggesting that merely planning a grip response primes the corresponding articulatory response. These results support the view that articulation and grasping are processed in a partially overlapping network.

Quantity language speakers show enhanced subcortical processing.

The complex auditory brainstem response (cABR) can reflect language-based plasticity in subcortical stages of auditory processing. It is sensitive to differences between language groups as well as stimulus properties, e.g. intensity or frequency. It is also sensitive to the synchronicity of the neural population stimulated by sound, which results in increased amplitude of wave V. Finnish is a full-fledged quantity language, in which word meaning is dependent upon duration of the vowels and consonants. Previous studies have shown that Finnish speakers have enhanced behavioural sound duration discrimination ability and larger cortical mismatch negativity (MMN) to duration change compared to German and French speakers. The next step is to find out whether these enhanced duration discrimination abilities of quantity language speakers originate at the brainstem level. Since German has a complementary quantity contrast which restricts the possible patterns of short and long vowels and consonants, the current experiment compared cABR between nonmusician Finnish and German native speakers using seven short complex stimuli. Finnish speakers had a larger cABR peak amplitude than German speakers, while the peak onset latency was only affected by stimulus intensity and spectral band. The results suggest that early cABR responses are better synchronised for Finns, which could underpin the enhanced duration sensitivity of quantity language speakers.

Excitability of hand motor areas during articulation of syllables.

It is known that articulating different syllables is linked to different grasp actions, e.g. [ti] is linked to precision grip, and [kɑ] to power grip. The aim of the present study was to test whether articulating or hearing these syllables would result in an increased activity in the representation of hand muscles involved in these two actions in a muscle-specific manner. To this end, we used transcranial magnetic stimulation (TMS) to investigate changes in the excitability of the left primary motor cortex (M1) innervating hand muscles while participants articulated or listened to meaningless syllables, listened to a metronome, or observed a fixation cross. The motor-evoked potentials of two hand muscles associated with either a precision or power grip exhibited significantly greater amplitudes during articulation than in passive listening, metronome, and fixation cross conditions. Moreover, these muscles exhibited similar patterns of excitability during articulation regardless of which syllable was articulated. The increased excitability of the left M1 hand area during articulation, but not during perception of the syllables, might be due to the cortico-cortical interaction between the motor representations of oral organs with the hand area.