Spatio-temporal distribution of brain activity associated with audio-visually congruent and incongruent speech and the McGurk Effect.

INTRODUCTION: Spatio-temporal distributions of cortical activity to audio-visual presentations of meaningless vowel-consonant-vowels and the effects of audio-visual congruence/incongruence, with emphasis on the McGurk effect, were studied. The McGurk effect occurs when a clearly audible syllable with one consonant, is presented simultaneously with a visual presentation of a face articulating a syllable with a different consonant and the resulting percept is a syllable with a consonant other than the auditorily presented one. METHODS: Twenty subjects listened to pairs of audio-visually congruent or incongruent utterances and indicated whether pair members were the same or not. Source current densities of event-related potentials to the first utterance in the pair were estimated and effects of stimulus-response combinations, brain area, hemisphere, and clarity of visual articulation were assessed. RESULTS: Auditory cortex, superior parietal cortex, and middle temporal cortex were the most consistently involved areas across experimental conditions. Early (<200 msec) processing of the consonant was overall prominent in the left hemisphere, except right hemisphere prominence in superior parietal cortex and secondary visual cortex. Clarity of visual articulation impacted activity in secondary visual cortex and Wernicke's area. McGurk perception was associated with decreased activity in primary and secondary auditory cortices and Wernicke's area before 100 msec, increased activity around 100 msec which decreased again around 180 msec. Activity in Broca's area was unaffected by McGurk perception and was only increased to congruent audio-visual stimuli 30-70 msec following consonant onset. CONCLUSIONS: The results suggest left hemisphere prominence in the effects of stimulus and response conditions on eight brain areas involved in dynamically distributed parallel processing of audio-visual integration. Initially (30-70 msec) subcortical contributions to auditory cortex, superior parietal cortex, and middle temporal cortex occur. During 100-140 msec, peristriate visual influences and Wernicke's area join in the processing. Resolution of incongruent audio-visual inputs is then attempted, and if successful, McGurk perception occurs and cortical activity in left hemisphere further increases between 170 and 260 msec.

Spatiotemporal distribution of cortical processing of first and second languages in bilinguals. II. Effects of phonologic and semantic priming.

This study determined the effects of phonology and semantics on the distribution of cortical activity to the second of a pair of words in first and second language (mixed pairs). The effects of relative proficiency in the two languages and linguistic setting (monolinguistic or mixed) are reported in a companion paper. Ten early bilinguals and 14 late bilinguals listened to mixed pairs of words in Arabic (L1) and Hebrew (L2) and indicated whether both words in the pair had the same or different meanings. The spatio-temporal distribution of current densities of event-related potentials were estimated for each language and according to semantic and phonologic relationship (same or different) compared with the first word in the pair. During early processing (<300 ms), brain activity in temporal and temporoparietal auditory areas was enhanced by phonologic incongruence between words in the pair and in Wernicke's area by both phonologic and semantic priming. In contrast, brain activities during late processing (>300 ms) were enhanced by semantic incongruence between the two words, particularly in temporal areas and in left hemisphere Broca's and Wernicke's areas. The latter differences were greater when words were in L2. Surprisingly, no significant effects of relative proficiency on processing the second word in the pair were found. These results indicate that the distribution of brain activity to the second of two words presented bilingually is affected differently during early and late processing by both semantic and phonologic priming by- and incongruence with the immediately preceding word.

Spatiotemporal distribution of cortical processing of first and second languages in bilinguals. I. Effects of proficiency and linguistic setting.

The study determined how spatiotemporal distribution of cortical activity to words in first and second language is affected by language, proficiency, and linguistic setting. Ten early bilinguals and 14 late adult bilinguals listened to pairs of words presented in Arabic (L1), Hebrew (L2), or in mixed pairs and indicated whether both words had the same meaning or not. Source current densities of event-related potentials were estimated. Activity to first words in the pair lateralized to right hemisphere, higher to L1 than L2 during early processing (<300 ms) among both groups but only among late bilinguals during late processing (>300 ms). During early and late processing, activities were larger in mixed than monolinguistic settings among early bilinguals but lower in mixed than in monolinguistic settings among late bilinguals. Late processing in auditory regions was of larger magnitude in left than right hemispheres among both groups. Activity to second words in the pair was larger in mixed than in monolinguistic settings during both early and late processing among both groups. Early processing of second words in auditory regions lateralized to the right among early bilinguals and to the left among late bilinguals, whereas late processing did not differ between groups. Wernicke's area activity during late processing of L2 was larger on the right, while on the left no significant differences between languages were found. The results show that cortical language processing in bilinguals differs between early and late processing and these differences are modulated by linguistic proficiency and setting.

A comparison of auditory evoked potentials to acoustic beats and to binaural beats.

The purpose of this study was to compare cortical brain responses evoked by amplitude modulated acoustic beats of 3 and 6 Hz in tones of 250 and 1000 Hz with those evoked by their binaural beats counterparts in unmodulated tones to indicate whether the cortical processes involved differ. Event-related potentials (ERPs) were recorded to 3- and 6-Hz acoustic and binaural beats in 2000 ms duration 250 and 1000 Hz tones presented with approximately 1 s intervals. Latency, amplitude and source current density estimates of ERP components to beats-evoked oscillations were determined and compared across beat types, beat frequencies and base (carrier) frequencies. All stimuli evoked tone-onset components followed by oscillations corresponding to the beat frequency, and a subsequent tone-offset complex. Beats-evoked oscillations were higher in amplitude in response to acoustic than to binaural beats, to 250 than to 1000 Hz base frequency and to 3 Hz than to 6 Hz beat frequency. Sources of the beats-evoked oscillations across all stimulus conditions located mostly to left temporal lobe areas. Differences between estimated sources of potentials to acoustic and binaural beats were not significant. The perceptions of binaural beats involve cortical activity that is not different than acoustic beats in distribution and in the effects of beat- and base frequency, indicating similar cortical processing.

Cortical evoked potentials to an auditory illusion: binaural beats.

OBJECTIVE: To define brain activity corresponding to an auditory illusion of 3 and 6Hz binaural beats in 250Hz or 1000Hz base frequencies, and compare it to the sound onset response. METHODS: Event-Related Potentials (ERPs) were recorded in response to unmodulated tones of 250 or 1000Hz to one ear and 3 or 6Hz higher to the other, creating an illusion of amplitude modulations (beats) of 3Hz and 6Hz, in base frequencies of 250Hz and 1000Hz. Tones were 2000ms in duration and presented with approximately 1s intervals. Latency, amplitude and source current density estimates of ERP components to tone onset and subsequent beats-evoked oscillations were determined and compared across beat frequencies with both base frequencies. RESULTS: All stimuli evoked tone-onset P(50), N(100) and P(200) components followed by oscillations corresponding to the beat frequency, and a subsequent tone-offset complex. Beats-evoked oscillations were higher in amplitude with the low base frequency and to the low beat frequency. Sources of the beats-evoked oscillations across all stimulus conditions located mostly to left lateral and inferior temporal lobe areas in all stimulus conditions. Onset-evoked components were not different across stimulus conditions; P(50) had significantly different sources than the beats-evoked oscillations; and N(100) and P(200) sources located to the same temporal lobe regions as beats-evoked oscillations, but were bilateral and also included frontal and parietal contributions. CONCLUSIONS: Neural activity with slightly different volley frequencies from left and right ear converges and interacts in the central auditory brainstem pathways to generate beats of neural activity to modulate activities in the left temporal lobe, giving rise to the illusion of binaural beats. Cortical potentials recorded to binaural beats are distinct from onset responses. SIGNIFICANCE: Brain activity corresponding to an auditory illusion of low frequency beats can be recorded from the scalp.

Auditory-evoked potentials to frequency increase and decrease of high- and low-frequency tones.

OBJECTIVE: To define cortical brain responses to large and small frequency changes (increase and decrease) of high- and low-frequency tones. METHODS: Event-Related Potentials (ERPs) were recorded in response to a 10% or a 50% frequency increase from 250 or 4000 Hz tones that were approximately 3 s in duration and presented at 500-ms intervals. Frequency increase was followed after 1 s by a decrease back to base frequency. Frequency changes occurred at least 1 s before or after tone onset or offset, respectively. Subjects were not attending to the stimuli. Latency, amplitude and source current density estimates of ERPs were compared across frequency changes. RESULTS: All frequency changes evoked components P(50), N(100), and P(200). N(100) and P(200) had double peaks at bilateral and right temporal sites, respectively. These components were followed by a slow negativity (SN). The constituents of N(100) were predominantly localized to temporo-parietal auditory areas. The potentials and their intracranial distributions were affected by both base frequency (larger potentials to low frequency) and direction of change (larger potentials to increase than decrease), as well as by change magnitude (larger potentials to larger change). The differences between frequency increase and decrease depended on base frequency (smaller difference to high frequency) and were localized to frontal areas. CONCLUSIONS: Brain activity varies according to frequency change direction and magnitude as well as base frequency. SIGNIFICANCE: The effects of base frequency and direction of change may reflect brain networks involved in more complex processing such as speech that are differentially sensitive to frequency modulations of high (consonant discrimination) and low (vowels and prosody) frequencies.

The auditory P50 component to onset and offset of sound.

OBJECTIVE: The auditory Event-Related Potentials (ERP) of component P50 to sound onset and offset have been reported to be similar, but their magnetic homologue has been reported absent to sound offset. We compared the spatio-temporal distribution of cortical activity during P50 to sound onset and offset, without confounds of spectral change. METHODS: ERPs were recorded in response to onsets and offsets of silent intervals of 0.5 s (gaps) appearing randomly in otherwise continuous white noise and compared to ERPs to randomly distributed click pairs with half second separation presented in silence. Subjects were awake and distracted from the stimuli by reading a complicated text. Measures of P50 included peak latency and amplitude, as well as source current density estimates to the clicks and sound onsets and offsets. RESULTS: P50 occurred in response to noise onsets and to clicks, while to noise offset it was absent. Latency of P50 was similar to noise onset (56 ms) and to clicks (53 ms). Sources of P50 to noise onsets and clicks included bilateral superior parietal areas. In contrast, noise offsets activated left inferior temporal and occipital areas at the time of P50. Source current density was significantly higher to noise onset than offset in the vicinity of the temporo-parietal junction. CONCLUSIONS: P50 to sound offset is absent compared to the distinct P50 to sound onset and to clicks, at different intracranial sources. P50 to stimulus onset and to clicks appears to reflect preattentive arousal by a new sound in the scene. Sound offset does not involve a new sound and hence the absent P50. SIGNIFICANCE: Stimulus onset activates distinct early cortical processes that are absent to offset.

The N1 complex to gaps in noise: effects of preceding noise duration and intensity.

OBJECTIVE: To study the effects of duration and intensity of noise that precedes gaps in noise on the N-Complex (N(1a) and N(1b)) of Event-Related Potentials (ERPs) to the gaps. METHODS: ERPs were recorded from 13 normal subjects in response to 20 ms gaps in 2-4.5 s segments of binaural white noise. Within each segment, the gaps appeared after 500, 1500, 2500 or 4000 ms of noise. Noise intensity was either 75, 60 or 45 dBnHL. Analysis included waveform peak measurements and intracranial source current density estimations, as well as statistical assessment of the effects of pre-gap noise duration and intensity on N(1a) and N(1b) and their estimated intracranial source activity. RESULTS: The N-Complex was detected at about 100 ms under all stimulus conditions. Latencies of N(1a) (at approximately 90 ms) and N(1b) (at approximately 150 ms) were significantly affected by duration of the preceding noise. Both their amplitudes and the latency of N(1b) were affected by the preceding noise intensity. Source current density was most prominent, under all stimulus conditions, in the vicinity of the temporo-parietal junction, with the first peak (N(1a)) lateralized to the left hemisphere and the second peak (N(1b)) - to the right. Additional sources with lower current density were more anterior, with a single peak spanning the duration of the N-Complex. CONCLUSIONS: The N(1a) and N(1b) of the N-Complex of the ERPs to gaps in noise are affected by both duration and intensity of the pre-gap noise. The minimum noise duration required for the appearance of a double-peaked N-Complex is just under 500 ms, depending on noise intensity. N(1a) and N(1b) of the N-Complex are generated predominantly in opposite temporo-parietal brain areas: N(1a) on the left and N(1b) on the right. SIGNIFICANCE: Duration and intensity interact to define the dual peaked N-Complex, signaling the cessation of an ongoing sound.

The composite N1 component to gaps in noise.

OBJECTIVE: To indicate whether the double peaked N(1) to gaps in continuous white noise is a composite of onset and offset responses to transients or whether it reflects higher processing such as change or mismatch detection and to assess the role of attention in this process. METHODS: Evoked potentials were recorded to two binaural stimulus types: (1) gaps of different durations randomly distributed in continuous white noise; and (2) click pairs at intervals identical to those between gap onsets and offsets in the continuous noise stimulus. Potentials to these stimuli were recorded while subjects read a text and while detecting gaps in noise or click pairs. RESULTS: Potentials were detected to all click pairs and to gaps of 5 ms or longer, corresponding to the subjects' psychoacoustic gap detection threshold. With long gap durations of 200-800 ms, distinct potentials to gap onset and gap offset were observed. The waveforms to all click pairs and to offsets of long gaps were similar and single-peaked, while potentials to gaps of 10 ms and longer, and potentials to onsets of long gaps were double-peaked, consisting of two N(1) negativities, 60 ms apart, irrespective of gap duration. The first (N(1a)), was more frontal in its distribution and similar to that of clicks. The second (N(1b)) peak's distribution was more central/temporal and its source locations and time course of activity were distinct. No effects of attention on any of the varieties and constituents of N(1) were observed. CONCLUSIONS: Comparing potentials to gap onsets, to click pairs and to gap offsets, suggests that potentials to gap onsets involve not only sound onset/offset responses (N(1), N(1a)) but also the subsequent pre-attentive perception of the cessation of an ongoing sound (N(1b)). We propose that N(1b) is distinct from change or mismatch detection and is associated with termination of an ongoing continuous stimulus. We propose to call it the N(egation)-process. SIGNIFICANCE: A constituent of the N(1) complex is shown to be associated with the pre-attentive perception of termination of an ongoing stimulus and to have distinct scalp distribution and intracranial sources.

Auditory middle-latency components to fusion of speech elements forming an auditory object.

OBJECTIVE: The purpose of this study was to define early brain activity associated with fusion of speech elements to form an auditory object in the middle-latency range preceding the F-Complex. METHODS: Stimuli were binaural formant transition and base, that were presented separately or fused to form the vowel-consonant-vowel sequence /ada/. Eleven right-handed, adult, native Hebrew speakers listened to 2/s presentations, and the brain potentials from C(z) during the 250 msec following transition onset (in the responses to transition and to the fused word) or following the time it would have been presented (in the response to base alone) were recorded. The net-fusion response was extracted by subtracting the sum of potentials to the base and the formant transition from the potentials to the fused sound. RESULTS: Auditory middle-latency components, comprising of 9 peaks and troughs were recorded in response to the base, to the formant transition and to the fused /ada/. In general, the responses to the fused object were significantly smaller in peak amplitude and in total activity (area under the curve) resulting in the difference waveform of the net-fusion response that also included 9 peaks, but with opposite polarities. CONCLUSIONS: The early middle-latency components to fusion indicate that the fusion of speech elements to a word involves inhibition, occlusion or both. The results are in line with the uniqueness of speech perception and the early role of the auditory cortex in speech analysis.