Audiovisual speech perception: Moving beyond McGurk.

Although it is clear that sighted listeners use both auditory and visual cues during speech perception, the manner in which multisensory information is combined is a matter of debate. One approach to measuring multisensory integration is to use variants of the McGurk illusion, in which discrepant auditory and visual cues produce auditory percepts that differ from those based on unimodal input. Not all listeners show the same degree of susceptibility to the McGurk illusion, and these individual differences are frequently used as a measure of audiovisual integration ability. However, despite their popularity, we join the voices of others in the field to argue that McGurk tasks are ill-suited for studying real-life multisensory speech perception: McGurk stimuli are often based on isolated syllables (which are rare in conversations) and necessarily rely on audiovisual incongruence that does not occur naturally. Furthermore, recent data show that susceptibility to McGurk tasks does not correlate with performance during natural audiovisual speech perception. Although the McGurk effect is a fascinating illusion, truly understanding the combined use of auditory and visual information during speech perception requires tasks that more closely resemble everyday communication: namely, words, sentences, and narratives with congruent auditory and visual speech cues.

Using the DRM paradigm to assess language processing in monolinguals and bilinguals.

Both languages are jointly activated in the bilingual brain, requiring bilinguals to select the target language while avoiding interference from the unwanted language. This cross-language interference is similar to the within-language interference created by the Deese-Roediger-McDermott false memory paradigm (DRM; Roediger & McDermott, 1995, Journal of Experimental Psychology: Learning, Memory, and Cognition, 21[4], 803-814). Although the mechanisms mediating false memory in the DRM paradigm remain an area of investigation, two of the more prominent theories-implicit associative response (IAR) and fuzzy trace-provide frameworks for using the DRM paradigm to advance our understanding of bilingual language processing. Three studies are reported comparing accuracy of monolingual and bilingual participants on different versions of the DRM. Study 1 presented lists of phonological associates and found that bilinguals showed higher rates of false recognition than did monolinguals. Study 2 used the standard semantic variant of the task and found that bilinguals showed lower false recognition rates than did monolinguals. Study 3 replicated and extended the findings in Experiment 2 in another semantic version of the task presented to younger and older adult monolingual and bilingual participants. These results are discussed within the frameworks of IAR and fuzzy-trace theories as further explicating differences between monolingual and bilingual processing.

Effects of age, word frequency, and noise on the time course of spoken word recognition.

This study assessed the effects of age, word frequency, and background noise on the time course of lexical activation during spoken word recognition. Participants (41 young adults and 39 older adults) performed a visual world word recognition task while we monitored their gaze position. On each trial, four phonologically unrelated pictures appeared on the screen. A target word was presented auditorily following a carrier phrase ("Click on the ________"), at which point participants were instructed to use the mouse to click on the picture that corresponded to the target word. High- and low-frequency words were presented in quiet to half of the participants. The other half heard the words in a low level of noise in which the words were still readily identifiable. Results showed that, even in the absence of phonological competitors in the visual array, high-frequency words were fixated more quickly than low-frequency words by both listener groups. Young adults were generally faster to fixate on targets compared to older adults, but the pattern of interactions among noise, word frequency, and listener age showed that older adults' lexical activation largely matches that of young adults in a modest amount of noise.

An age-related deficit in resolving interference: Evidence from speech perception.

The presence of noise and interfering information can pose major difficulties during speech perception, particularly for older adults. Analogously, interference from similar representations during retrieval is a major cause of age-related memory failures. To demonstrate a suppression mechanism that underlies such speech and memory difficulties, we tested the hypothesis that interference between targets and competitors is resolved by suppressing competitors, thereby rendering them less intelligible in noise. In a series of experiments using a paradigm adapted from Healey, Hasher, and Campbell (2013), we presented a list of words that included target/competitor pairs of orthographically similar words (e.g., ALLERGY and ANALOGY). After a delay, participants solved fragments (e.g., A_L__GY), some of which resembled both members of the target/competitor pair, but could only be completed by the target. We then assessed the consequence of having successfully resolved this interference by asking participants to identify words in noise, some of which included the rejected competitor words from the previous phase. Consistent with a suppression account of interference resolution, younger adults reliably demonstrated reduced identification accuracy for competitors, indicating that they had effectively rejected, and therefore suppressed, competitors. In contrast, older adults showed a relative increase in accuracy for competitors relative to young adults. Such results suggest that older adults' reduced ability to suppress these representations resulted in sustained access to lexical traces, subsequently increasing perceptual identification of such items. We discuss these findings within the framework of inhibitory control theory in cognitive aging and its implications for age-related changes in speech perception. (PsycINFO Database Record

Age-related differences in inhibitory control predict audiovisual speech perception.

Audiovisual (AV) speech perception is the process by which auditory and visual sensory signals are integrated and used to understand what a talker is saying during face-to-face communication. This form of communication is markedly superior to speech perception in either sensory modality alone. However, there are additional lexical factors that are affected by age-related cognitive changes that may contribute to differences in AV perception. In the current study, we extended an existing model of spoken word identification to the AV domain, and examined the cognitive factors that contribute to age-related and individual differences in AV perception of words varying in lexical difficulty (i.e., on the basis of competing items). Young (n = 49) and older adults (n = 50) completed a series of cognitive inhibition tasks and a spoken word identification task. The words were presented in auditory-only, visual-only, and AV conditions, and were equally divided into lexically hard (words with many competitors) and lexically easy (words with few competitors). Overall, young adults demonstrated better inhibitory abilities and higher identification performance than older adults. However, whereas no relationship was observed between inhibitory abilities and AV word identification performance in young adults, there was a significant relationship between Stroop interference and AV identification of lexically hard words in older adults. These results are interpreted within the framework of existing models of spoken-word recognition with implications for how cognitive deficits in older adults contribute to speech perception.