Congruency of auditory sounds and visual letters modulates mismatch negativity and P300 event-related potentials.

A key determinant of skilled reading is the ability to integrate the orthographic and auditory forms of language. A number of prior studies have identified neural markers in adult readers corresponding to audio-visual integration of letters and their corresponding sounds. However, there remains some controversy as to the stage of processing at which this occurs. In the present study, we examined this issue using event-related potentials (ERPs), due to their sensitivity to the timing of perceptual and cognitive processes. Letter sounds were presented auditorily in an unattended mismatch negativity (MMN) paradigm, which is argued to be indicative of auditory sensory memory. Concurrently, participants performed a visual letter identification task. On critical trials, the auditory stimulus was played concurrently with the visual letters. We observed significant MMNs both when the visual letter was congruent with the auditory stimulus, and when it was incongruent. However, the magnitude and scalp distribution of this effect was attenuated in incongruent trials. We also observed a later-going effect of congruency on P300 trials, marked by increased amplitudes and latencies for incongruent compared to congruent trials. The results suggest audiovisual integration of letters and sounds can and does occur during relatively early pre-attentive stages of sensory processing, and that these effects extend to later-going attentional phases of processing as well.

Effect of short and long exposure duration and dual-tasking on a global-local task.

Past research has demonstrated a global advantage in responses to visually presented hierarchical stimuli such that, on incongruent trials, the global form interferes with responses to the local level [Kimchi, R. (1992). Primacy of wholistic processing and global/local paradigm: A critical review. Psychological Bulletin, 112, 24-38]. In Experiment 1, 32 adults performed alternating blocks of global or local identification of hierarchical letter stimuli in which the global and local letters were congruent, incongruent, or neutral, and were presented at either a short (17 ms) or long (100 ms) exposure duration. A global advantage was demonstrated at both durations. In the local-directed task, interference on incongruent, relative to neutral, trials was observed at both exposure durations, but facilitation on congruent trials, relative to neutral trials, was present only when stimuli were presented at the long exposure duration. In Experiment 2, global or local identification was performed by another group of 24 adults at either a long or short exposure duration, and also under conditions of full attention (FA) or dual-task (DT) conditions with a digit-monitoring task. Under FA, we again found significant interference at both exposure durations, but facilitation only at the long exposure duration. Under DT conditions, the pattern of facilitation and interference at the short duration remained unchanged. At the long duration, however, dual-tasking eliminated interference in the RT but not error data, while facilitation was present in both sets of data. Results are in line with a perceptual account of the global advantage, and suggest that facilitation requires consciously-mediated processes, whereas interference does not.