Relationship Between Cognitive Functions and Multilevel Language Processing: Data From Chinese Speakers With Aphasia and Implications.

PURPOSE: This study aims to investigate the relationship between nonverbal cognitive functions and language processing of people with aphasia (PWA) by taking a data-driven approach, as well as multiple cognitive components and multilevel linguistic perspectives. It is hypothesized that language performance is differentially associated with cognitive processing of PWA, with executive functions (EFs) playing a stronger role in language tasks with increasing linguistic complexity. METHOD: A language battery assessing word comprehension/production, sentence comprehension, and discourse production, together with a series of nonlinguistic cognitive tasks targeting simple/complex attention, short-term/working memory, or EFs, was administered to 53 Cantonese-speaking PWA. Cognitive factors extracted from principal component analysis applied to the cognitive battery served as predictors in four multiple regression analyses to predict PWA's performance at various linguistic levels. RESULTS: Two cognitive factors, representing (a) simple attention and memory and (b) EF, were extracted. The former predicted performance in word processing tasks, whereas EF significantly predicted performance in all language tasks with increasing contribution as a function of linguistic complexity. CONCLUSION: The results based on Chinese PWA provide comprehensive evidence for the view that language performance is the end product of interaction between linguistic and nonlinguistic functions and have clear implications for clinical management of PWA.

Dissociation of tone merger and congenital amusia in Hong Kong Cantonese.

While the issue of individual variation has been widely studied in second language learning or processing, it is less well understood how perceptual and musical aptitude differences can explain individual variation in native speech processing. In the current study, we make use of tone merger in Hong Kong Cantonese, an ongoing sound change that concerns the merging of tones in perception, production or both in a portion of native speakers, to examine the possible relationship between tone merger and musical and pitch abilities. Although a previous study has reported the occurrence of tone merger independently of musical training, it has not been investigated before whether tone-merging individuals, especially those merging tones in perception, would have inferior musical perception and fine-grained pitch sensitivities, given the close relationship of speech and music. To this end, we tested three groups of tone-merging individuals with various tone perception and production profiles on musical perception and pitch threshold tasks, in comparison to a group of Cantonese speakers with congenital amusia, and another group of controls without tone merger or amusia. Additionally, the amusics were compared with tone-merging individuals on the details of their tone discrimination and production profiles. The results showed a clear dissociation of tone merger and amusia, with the tone-merging individuals exhibiting intact musical and pitch abilities; on the other hand, the amusics demonstrated widespread difficulties in tone discrimination yet intact tone production, in contrast to the highly selective confusion of a specific tone pair in production or discrimination in tone-merging individuals. These findings provide the first evidence that tone merger and amusia are distinct from each other, and further suggest that the cause of tone merger may lie elsewhere rather than being driven by musical or pitch deficits. We also discussed issues arising from the current findings regarding the neural mechanisms of tone merger and amusia.

N170 reflects orthographic uniqueness point effects in English among native Japanese and Korean readers.

Orthographic uniqueness point (OUP) refers to the letter position of a word at which it is distinguishable from other lexical items in the language. Previous findings of OUP effects have been mixed and mainly demonstrated in native readers of alphabetic languages. The current study investigated whether OUP effects could be shown among non-native readers in a visual repetition detection task. The experiment tested three OUP conditions (early, mid, late) in native English readers and proficient non-native English readers whose native scripts were Japanese or Korean. Results revealed main effects of OUP on N170 amplitude, where early OUP words elicited more negative N170 and late OUP words elicited marginally less negative N170 than mean activation for both native and non-native readers. There was no indication that non-linearity or non-alphabetic nature of one's native script influenced OUP effects. Results were consistent with a parallel letter processing account in single word reading.

Functional orthographic units in Chinese character reading: Are there abstract radical identities?

Previous research has shown that the components of Chinese characters (e.g., semantic components, phonetic components, and radicals) serve as processing units in reading. One outstanding question concerns the existence of amodal orthographic representations that unify multiple, form-specific character components, similar to the abstract letter identities (ALIs) that unify case-specific letter forms (A/a) in Roman script. Although Chinese does not have case, a subset of semantic radicals have multiple forms (e.g., - are both "water" radicals), allowing for a test of the existence of Abstract Radical Identities (ARIs) that unify the multiple forms. In Experiment 1, a visual same-different judgement task was used to detect the presence of ARI representations. Evidence for ARIs was provided by the finding that radical pairs with different forms but the same radical identity were judged to be visually different more slowly than matched pairs of different forms with different radical identities. In Experiment 2, we evaluated ARI effects in real character reading. A lexical decision priming task compared prime-target character pairs containing radicals with the same identity but different forms (e.g., -) with matched prime-target character pairs with unrelated radicals (e.g., -). Inhibitory priming was observed only in the same-identity radical condition compared with the unrelated condition. These combined results provide, for the first time, evidence of format-free representations of orthographic units in Chinese characters-abstract radical identities (ARIs).

Top-down and bottom-up mechanisms as reflected by beta and gamma oscillations in speech perception: An individual-difference approach.

Recent neurophysiological studies have proposed distinct roles of ß and γ oscillations in implementing top-down and bottom-up processes. The present study aims to test this hypothesis in the domain of speech perception. We examined ß and γ oscillations elicited to a tone contrast in a passive oddball paradigm, and their relationships with discrimination sensitivity d' and RT from two groups of healthy adults who showed high and low discrimination sensitivity to the contrast. The low-sensitivity group showed a significant reduction in ß, which was further related to d'. Individual differences in RT were related to different frequency bands in the two groups, with a RT-ß correlation in the low-sensitivity group, and a RT-γ relation in the high-sensitivity group. Based on these findings, we suggest that ß, implicated in top-down processing, reflects individual differences in phonological representations, and that γ, involved in bottom-up processing, reflects individual differences in acoustic encoding.

Interactions of age of acquisition and lexical frequency effects with phonological regularity: An ERP study.

Across languages, lexical frequency and age of acquisition (AoA) are important predictors of word reading that are naturally correlated, and both variables have been shown to interact with phonological regularity. Previous behavioral findings in Chinese have shown stronger regularity effects in low frequency relative to high frequency characters. Meanwhile, the arbitrary mapping hypothesis predicts stronger interference effects in late-acquired irregular words. This study examined the neural bases of these phenomena in Chinese using a delayed naming task with 480 single characters. Single-trial ERP analyses were conducted to contrast the interactions of frequency or AoA effects with phonological regularity, while controlling for the other factor. Results showed differential and significant effects of AoA and frequency at the N400 and late positive component (LPC), respectively, indicating their independence. ERP interaction patterns of frequency and regularity in Chinese were observed for the first time in the LPC time window and suggested semantic interference from sublexical units during character reading. Interaction of AoA and regularity at the N400 could be explained by the semantic hypothesis but appeared inconsistent with the arbitrary mapping hypothesis.

Use of co-verbal gestures during word-finding difficulty among Cantonese speakers with fluent aphasia and unimpaired controls.

BACKGROUND: Co-verbal gestures refer to hand or arm movements made during speaking. Spoken language and gestures have been shown to be tightly integrated in human communication. AIMS: The present study investigated whether co-verbal gesture use was associated with lexical retrieval in connected speech in unimpaired speakers and persons with aphasia (PWA). METHODS & PROCEDURES: Narrative samples of 58 fluent PWA and 58 control speakers were extracted from Cantonese AphasiaBank. Based on the indicators of word-finding difficulty (WFD) in connected speech adapted from previous research, and a gesture annotation system with independent coding of gesture forms and functions, all WFD instances were identified. The presence and type of gestures accompanying each incident of WFD were then annotated. Finally, whether the use of gesture was accompanied by resolution of WFD, i.e., the corresponding target word could be retrieved, was examined. OUTCOMES & RESULTS: Employment of co-verbal gesture did not seem to be related to the success of word retrieval. PWA's naming ability at single-word level and their overall language ability (as reflected by the aphasia quotient of the Cantonese version of the Western Aphasia Battery) were found to be the two strongest predictors of success rate of resolving WFD. CONCLUSIONS: The Lexical Retrieval Hypothesis highlighting the facilitative functions of iconic and metaphoric gestures in lexical retrieval was not supported. Challenges in conducting research related to WFD, and the clinical implications in gesture-based language intervention for PWA were discussed.

Cantonese AphasiaBank: An annotated database of spoken discourse and co-verbal gestures by healthy and language-impaired native Cantonese speakers.

This article reports the construction of a multimodal annotated database of spoken discourse and co-verbal gestures by native healthy speakers of Cantonese and individuals with language impairment: the Cantonese AphasiaBank. This corpus was established as a foundation for aphasiologists and clinicians to use in designing and conducting research investigations into theoretical and clinical issues related to acquired language disorders in Chinese. Details in terms of the purpose, structure, and levels of annotation of the database (containing part-of-speech-annotated orthographic transcripts with Romanization and the corresponding videos) are described. The discussion presents the challenges of building a spoken database of a language that is not linguistically well-researched and that does not have a standardized written form for many of its lexical items, as well as presenting how these issues were addressed. Most importantly, the article highlights the potential of Cantonese AphasiaBank as a powerful research tool for linguists and psycholinguists.

Impact of characteristics of L1 literacy experience on picture processing: ERP data from trilingual non-native Chinese and English readers.

Previous studies involving ex-illiterate and young readers of alphabetic scripts have shown that processing of non-verbal visual stimuli may be affected by literacy, which is attributable to intensive perceptual training during reading acquisition. This study examined whether the characteristics of one's native writing system, with respect to visual complexity and overall shape of orthographic unit, would influence the processing of pictured objects using event-related potential (ERP) with linear mixed-effects modeling. Kanji and Hangul constitute an interesting contrast as they differ in visual complexity but are similar in orthographic shape. Neural responses to pictures were analyzed in N170 and N400 reflecting ease of recognition and access to semantic representation, respectively. Trilingual speakers with Japanese or Korean as L1, who learned to read Chinese and English as non-native languages, as well as native Chinese and native English readers participated in a repetition detection task in which mixed blocks of pictures and words (Chinese or English) were presented. The overall results showed bilaterally distributed N170 and N400, and group differences in the N400 were confined to the anterior region. More importantly, Japanese readers exhibited more negative N170 and less negative N400 than Korean participants regardless of language context. The present findings have provided insights into the possible impact of reading acquisition on non-linguistic visual processing, and suggest that one's early experience of a visually complex orthography has positive transfer to processing of line drawings in terms of more efficient visual recognition and semantic access.

Induced gamma oscillations index individual differences in speech sound perception and production.

Auditory neuroscience has provided strong evidence that neural oscillations synchronize to the rhythm of speech stimuli, and oscillations at different frequencies have been linked to processing of different language structures. The present study aims to examine how these ubiquitous neurophysiological attributes may inform us about the brain processes that underpin individual differences in speech perception and production, which in turn elucidate the specific functions of neural oscillations in the domain of speech processing. To this end, we recorded electrophysiological responses to a lexical tone contrast in a passive auditory oddball paradigm from two groups of healthy tone-language speakers who were equal in perceptual discriminability but differed in response latency and production distinctiveness of the tone contrast. Time-frequency analysis was applied to the EEG data, and decomposed into theta (4-7 Hz), beta (12-30 Hz), and gamma (30-50 Hz) frequency bands. Results show that listeners with longer discrimination RT and less distinctive production showed significantly higher induced (non-phase-locked) gamma during tone processing. Moreover, among speakers with less distinctive production, individual differences in induced gamma were significantly correlated with discrimination latency and production distinction. Based on the present findings, we propose that differences in gamma oscillations reflect differential sensory/perceptual computations during acoustic encoding, impacting the quality of perceptual representations, which further mediates individual differences in speech perception and production.

Measuring discourse coherence in anomic aphasia using Rhetorical Structure Theory.

PURPOSE: The existing body of work regarding discourse coherence in aphasia has provided mixed results, leaving the question of coherence being impaired or intact as a result of brain injury unanswered. In this study, discourse coherence in non-brain-damaged (NBD) speakers and speakers with anomic aphasia was investigated quantitatively and qualitatively. METHOD: Fifteen native speakers of Cantonese with anomic aphasia and 15 NBD participants produced 60 language samples. Elicitation tasks included story-telling induced by a picture series and a procedural description. The samples were annotated for discourse structure in the framework of Rhetorical Structure Theory (RST) in order to analyse a number of structural parameters. After that 20 naïve listeners rated coherence of each sample. RESULT: Disordered discourse was rated as significantly less coherent. The NBD group demonstrated a higher production fluency than the participants with aphasia and used a richer set of semantic relations to create discourse, particularly in the description of settings, expression of causality, and extent of elaboration. People with aphasia also tended to omit essential information content. CONCLUSION: Reduced essential information content, lower degree of elaboration, and a larger amount of structural disruptions may have contributed to the reduced overall discourse coherence in speakers with anomic aphasia.

An analysis of topics and vocabulary in Chinese oral narratives by normal speakers and speakers with fluent aphasia.

This study analysed the topic and vocabulary of Chinese speakers based on language samples of personal recounts in a large spoken Chinese database recently made available in the public domain, i.e. Cantonese AphasiaBank ( http://www.speech.hku.hk/caphbank/search/ ). The goal of the analysis is to offer clinicians a rich source for selecting ecologically valid training materials for rehabilitating Chinese-speaking people with aphasia (PWA) in the design and planning of culturally and linguistically appropriate treatments. Discourse production of 65 Chinese-speaking PWA of fluent types (henceforth, PWFA) and their non-aphasic controls narrating an important event in their life were extracted from Cantonese AphasiaBank. Analyses of topics and vocabularies in terms of part-of-speech, word frequency, lexical semantics, and diversity were conducted. There was significant overlap in topics between the two groups. While the vocabulary was larger for controls than that of PWFA as expected, they were similar in distribution across parts-of-speech, frequency of occurrence, and the ratio of concrete to abstract items in major open word classes. Moreover, proportionately more different verbs than nouns were employed at the individual level for both speaker groups. The findings provide important implications for guiding directions of aphasia rehabilitation not only of fluent but also non-fluent Chinese aphasic speakers.

A Comparison of Coverbal Gesture Use in Oral Discourse Among Speakers With Fluent and Nonfluent Aphasia.

Purpose: Coverbal gesture use, which is affected by the presence and degree of aphasia, can be culturally specific. The purpose of this study was to compare gesture use among Cantonese-speaking individuals: 23 neurologically healthy speakers, 23 speakers with fluent aphasia, and 21 speakers with nonfluent aphasia. Method: Multimedia data of discourse samples from these speakers were extracted from the Cantonese AphasiaBank. Gestures were independently annotated on their forms and functions to determine how gesturing rate and distribution of gestures differed across speaker groups. A multiple regression was conducted to determine the most predictive variable(s) for gesture-to-word ratio. Results: Although speakers with nonfluent aphasia gestured most frequently, the rate of gesture use in counterparts with fluent aphasia did not differ significantly from controls. Different patterns of gesture functions in the 3 speaker groups revealed that gesture plays a minor role in lexical retrieval whereas its role in enhancing communication dominates among the speakers with aphasia. The percentages of complete sentences and dysfluency strongly predicted the gesturing rate in aphasia. Conclusions: The current results supported the sketch model of language-gesture association. The relationship between gesture production and linguistic abilities and clinical implications for gesture-based language intervention for speakers with aphasia are also discussed.

A Quantitative Study of Right Dislocation in Cantonese Spoken Discourse.

Right Dislocation (RD) has been suggested to be a focus marking device carrying an affective function motivated by limited planning time in conversation. The current study investigated the effects of genre type, planning load and affective function on the use of RD in Cantonese monologues. Discourse data were extracted from a recently developed corpus of oral narratives in Cantonese Chinese containing language samples from 144 native Cantonese speakers evenly distributed in age, education levels and gender. Three genre types representing different structures, styles and degrees of topic familiarity were chosen for an RD analysis: procedural description, story-telling and recount of personal event. The results revealed that genre types and planning load influenced the rate of RD occurrence. (1) Specifically, the lowest proportion of RD occurred in procedural description, assumed to be the most structured genre; whereas the highest rate was found in personal event recount, considered to be the most stylized and less structured genre. (2) The highest proportion of RD appeared near the end of a narrative, where heavier cognitive load is demanded compared with the beginning of a narrative; moreover, RD also tended to co-occur with disfluency. (3) There was a high percentage of RD tokens in the personal event recount for expressing explicit emotions; and (4) a lower rate of occurrence of RD was found in monologues than previous studies based on conversations. The overall findings suggest that the use of RD is sensitive to genre structure and style, as well as planning load effects.

Cognitive basis of individual differences in speech perception, production and representations: The role of domain general attentional switching.

This study investigated whether individual differences in cognitive functions, attentional abilities in particular, were associated with individual differences in the quality of phonological representations, resulting in variability in speech perception and production. To do so, we took advantage of a tone merging phenomenon in Cantonese, and identified three groups of typically developed speakers who could differentiate the two rising tones (high and low rising) in both perception and production [+Per+Pro], only in perception [+Per-Pro], or in neither modalities [-Per-Pro]. Perception and production were reflected, respectively, by discrimination sensitivity d' and acoustic measures of pitch offset and rise time differences. Components of event-related potential (ERP)-the mismatch negativity (MMN) and the ERPs to amplitude rise time-were taken to reflect the representations of the acoustic cues of tones. Components of attention and working memory in the auditory and visual modalities were assessed with published test batteries. The results show that individual differences in both perception and production are linked to how listeners encode and represent the acoustic cues (pitch contour and rise time) as reflected by ERPs. The present study has advanced our knowledge from previous work by integrating measures of perception, production, attention, and those reflecting quality of representation, to offer a comprehensive account for the underlying cognitive factors of individual differences in speech processing. Particularly, it is proposed that domain-general attentional switching affects the quality of perceptual representations of the acoustic cues, giving rise to individual differences in perception and production.

Individual differences in processing pitch contour and rise time in adults: A behavioral and electrophysiological study of Cantonese tone merging.

One way to understand the relationship between speech perception and production is to examine cases where the two dissociate. This study investigates the hypothesis that perceptual acuity reflected in event-related potentials (ERPs) to rise time of sound amplitude envelope and pitch contour [reflected in the mismatch negativity (MMN)] may associate with individual differences in production among speakers with otherwise comparable perceptual abilities. To test this hypothesis, advantage was taken of an on-going sound change-tone merging in Cantonese, and compared the ERPs between two groups of typically developed native speakers who could discriminate the high rising and low rising tones with equivalent accuracy but differed in the distinctiveness of their production of these tones. Using a passive oddball paradigm, early positive-going EEG components to rise time and MMN to pitch contour were elicited during perception of the two tones. Significant group differences were found in neural responses to rise time rather than pitch contour. More importantly, individual differences in efficiency of tone discrimination in response latency and magnitude of neural responses to rise time were correlated with acoustic measures of F0 offset and rise time differences in productions of the two rising tones.

Electrophysiological evidence of sublexical phonological access in character processing by L2 Chinese learners of L1 alphabetic scripts.

While Chinese character reading relies more on addressed phonology relative to alphabetic scripts, skilled Chinese readers also access sublexical phonological units during recognition of phonograms. However, sublexical orthography-to-phonology mapping has not been found among beginning second language (L2) Chinese learners. This study investigated character reading in more advanced Chinese learners whose native writing system is alphabetic. Phonological regularity and consistency were examined in behavioral responses and event-related potentials (ERPs) in lexical decision and delayed naming tasks. Participants were 18 native English speakers who acquired written Chinese after age 5 years and reached grade 4 Chinese reading level. Behaviorally, regular characters were named more accurately than irregular characters, but consistency had no effect. Similar to native Chinese readers, regularity effects emerged early with regular characters eliciting a greater N170 than irregular characters. Regular characters also elicited greater frontal P200 and smaller N400 than irregular characters in phonograms of low consistency. Additionally, regular-consistent characters and irregular-inconsistent characters had more negative amplitudes than irregular-consistent characters in the N400 and LPC time windows. The overall pattern of brain activities revealed distinct regularity and consistency effects in both tasks. Although orthographic neighbors are activated in character processing of L2 Chinese readers, the timing of their impact seems delayed compared with native Chinese readers. The time courses of regularity and consistency effects across ERP components suggest both assimilation and accommodation of the reading network in learning to read a typologically distinct second orthographic system.

Co-verbal gestures among speakers with aphasia: Influence of aphasia severity, linguistic and semantic skills, and hemiplegia on gesture employment in oral discourse.

UNLABELLED: The use of co-verbal gestures is common in human communication and has been reported to assist word retrieval and to facilitate verbal interactions. This study systematically investigated the impact of aphasia severity, integrity of semantic processing, and hemiplegia on the use of co-verbal gestures, with reference to gesture forms and functions, by 131 normal speakers, 48 individuals with aphasia and their controls. All participants were native Cantonese speakers. It was found that the severity of aphasia and verbal-semantic impairment was associated with significantly more co-verbal gestures. However, there was no relationship between right-sided hemiplegia and gesture employment. Moreover, significantly more gestures were employed by the speakers with aphasia, but about 10% of them did not gesture. Among those who used gestures, content-carrying gestures, including iconic, metaphoric, deictic gestures, and emblems, served the function of enhancing language content and providing information additional to the language content. As for the non-content carrying gestures, beats were used primarily for reinforcing speech prosody or guiding speech flow, while non-identifiable gestures were associated with assisting lexical retrieval or with no specific functions. The above findings would enhance our understanding of the use of various forms of co-verbal gestures in aphasic discourse production and their functions. Speech-language pathologists may also refer to the current annotation system and the results to guide clinical evaluation and remediation of gestures in aphasia. LEARNING OUTCOMES: None.

Relationship between individual differences in speech processing and cognitive functions.

A growing body of research has suggested that cognitive abilities may play a role in individual differences in speech processing. The present study took advantage of a widespread linguistic phenomenon of sound change to systematically assess the relationships between speech processing and various components of attention and working memory in the auditory and visual modalities among typically developed Cantonese-speaking individuals. The individual variations in speech processing are captured in an ongoing sound change-tone merging in Hong Kong Cantonese, in which typically developed native speakers are reported to lose the distinctions between some tonal contrasts in perception and/or production. Three groups of participants were recruited, with a first group of good perception and production, a second group of good perception but poor production, and a third group of good production but poor perception. Our findings revealed that modality-independent abilities of attentional switching/control and working memory might contribute to individual differences in patterns of speech perception and production as well as discrimination latencies among typically developed speakers. The findings not only have the potential to generalize to speech processing in other languages, but also broaden our understanding of the omnipresent phenomenon of language change in all languages.

A Coding System with Independent Annotations of Gesture Forms and Functions during Verbal Communication: Development of a Database of Speech and GEsture (DoSaGE).

Gestures are commonly used together with spoken language in human communication. One major limitation of gesture investigations in the existing literature lies in the fact that the coding of forms and functions of gestures has not been clearly differentiated. This paper first described a recently developed Database of Speech and GEsture (DoSaGE) based on independent annotation of gesture forms and functions among 119 neurologically unimpaired right-handed native speakers of Cantonese (divided into three age and two education levels), and presented findings of an investigation examining how gesture use was related to age and linguistic performance. Consideration of these two factors, for which normative data are currently very limited or lacking in the literature, is relevant and necessary when one evaluates gesture employment among individuals with and without language impairment. Three speech tasks, including monologue of a personally important event, sequential description, and story-telling, were used for elicitation. The EUDICO Linguistic ANnotator (ELAN) software was used to independently annotate each participant's linguistic information of the transcript, forms of gestures used, and the function for each gesture. About one-third of the subjects did not use any co-verbal gestures. While the majority of gestures were non-content-carrying, which functioned mainly for reinforcing speech intonation or controlling speech flow, the content-carrying ones were used to enhance speech content. Furthermore, individuals who are younger or linguistically more proficient tended to use fewer gestures, suggesting that normal speakers gesture differently as a function of age and linguistic performance.