Evaluating perceptual integration: uniting response-time- and accuracy-based methodologies.

This investigation brings together a response-time system identification methodology (e.g., Townsend & Wenger Psychonomic Bulletin & Review 11, 391-418, 2004a) and an accuracy methodology, intended to assess models of integration across stimulus dimensions (features, modalities, etc.) that were proposed by Shaw and colleagues (e.g., Mulligan & Shaw Perception & Psychophysics 28, 471-478, 1980). The goal was to theoretically examine these separate strategies and to apply them conjointly to the same set of participants. The empirical phases were carried out within an extension of an established experimental design called the double factorial paradigm (e.g., Townsend & Nozawa Journal of Mathematical Psychology 39, 321-359, 1995). That paradigm, based on response times, permits assessments of architecture (parallel vs. serial processing), stopping rule (exhaustive vs. minimum time), and workload capacity, all within the same blocks of trials. The paradigm introduced by Shaw and colleagues uses a statistic formally analogous to that of the double factorial paradigm, but based on accuracy rather than response times. We demonstrate that the accuracy measure cannot discriminate between parallel and serial processing. Nonetheless, the class of models supported by the accuracy data possesses a suitable interpretation within the same set of models supported by the response-time data. The supported model, consistent across individuals, is parallel and has limited capacity, with the participants employing the appropriate stopping rule for the experimental setting.

Processing of invisible social cues.

Successful interactions between people are dependent on rapid recognition of social cues. We investigated whether head direction--a powerful social signal--is processed in the absence of conscious awareness. We used continuous flash interocular suppression to render stimuli invisible and compared the reaction time for face detection when faces were turned towards the viewer and turned slightly away. We found that faces turned towards the viewer break through suppression faster than faces that are turned away, regardless of eye direction. Our results suggest that detection of a face with attention directed at the viewer occurs even in the absence of awareness of that face. While previous work has demonstrated that stimuli that signal threat are processed without awareness, our data suggest that the social relevance of a face, defined more broadly, is evaluated in the absence of awareness.

Attentive and pre-attentive processes in change detection and identification.

In studies of change blindness, observers often have the phenomenological impression that the blindness is overcome all at once, so that change detection, localization and identification apparently occur together. Three experiments are described that explore dissociations between these processes using a discrete trial procedure in which 2 visual frames are presented sequentially with no intervening inter-frame-interval. The results reveal that change detection and localization are essentially perfect under these conditions regardless of the number of elements in the display, which is consistent with the idea that change detection and localization are mediated by pre-attentive parallel processes.In contrast, identification accuracy for an item before it changes is generally poor, and is heavily dependent on the number of items displayed. Identification accuracy after a change is substantially better, but depends on the new item's duration. This suggests that the change captures attention, which substantially enhances the likelihood of correctly identifying the new item. However, the results also reveal a limited capacity to identify unattended items. Specifically, we provide evidence that strongly suggests that, at least under these conditions, observers were able to identify two items without focused attention. Our results further suggest that spatial pre-cues that attract attention to an item before the change occurs simply ensure that the cued item is one of the two whose identity is encoded.

On the minimization of task switch costs following long-term training.

Flexible, context-dependent linkages between stimulus and response are fundamental to adaptive behavior. In the present article, we evaluate the limits of this flexibility by exploring the asymptotic efficiency of people's ability to switch between two different sensorimotor mappings. Two stimulus-response (S-R) mappings were learned, either both on the same hand (unimanual condition) or one mapping per hand (intermanual condition). The S-R mappings presented bivalent stimuli and employed the same response keys. A novel training regimen successfully reduced task-switch costs to approximately 20 msec, suggesting that residual switch costs cannot be eliminated. These costs cannot be entirely attributed to the cognitive control process of task-set reconfiguration, because they are observed over cued switch intervals of several seconds. Two additional issues in motor learning were addressed: the single or dual loci of manual motor control and the coordinate system of task representation. First, the results favored the notion of independent controllers for each hand instead of a single dominant controller, since intermanual performance was superior to unimanual performance. Second, a transfer task tested internal (egocentric) and external (allocentric) coordinate systems. Transfer was more effective using the external coordinate system, suggesting that the S-R mappings reflected the association between the bivalent stimuli and external goals (i.e., the individual keys) rather than the concrete pattern of muscle contractions (i.e., the finger pressing the key). Finally, retention tests revealed that these learned S-R associations were remarkably durable, since no decrements in fluent task-switching performance were observed after 10 months without practice.

Failures to see: attentive blank stares revealed by change blindness.

Change blindness illustrates a remarkable limitation in visual processing by demonstrating that substantial changes in a visual scene can go undetected. Because these changes can ultimately be detected using top-down driven search processes, many theories assign a central role to spatial attention in overcoming change blindness. Surprisingly, it has been reported that change blindness can occur during blink-contingent changes even when observers fixate the changing location [O'Regan, J. K., Deubel, H., Clark, J. J., & Rensink, R. A. (2000). Picture changes during blinks: Looking without seeing and seeing without looking. Visual Cognition, 7, 191-212]. However, eye blinks produce a transient disruption of vision that is independent of any associated changes in the retinal image. We determined whether these 'attentive blank stares' could occur in the absence of blink-mediated visual suppression. Using a flicker change-blindness paradigm we confirm that despite direct attentive fixations, obvious scene changes often remain undetected. We conclude that change detection involves object or feature based attentional mechanisms, which can be 'misdirected' despite the allocation of spatial attention to the position of the change.

Multimodal access to verbal name codes.

Congruent information conveyed over different sensory modalities often facilitates a variety of cognitive processes, including speech perception (Sumby & Pollack, 1954). Since auditory processing is substantially faster than visual processing, auditory-visual integration can occur over a surprisingly wide temporal window (Stein, 1998). We investigated the processing architecture mediating the integration of acoustic digit names with corresponding symbolic visual forms. The digits "1" or "2" were presented in auditory, visual, or bimodal format at several stimulus onset asynchronies (SOAs; 0, 75, 150, and 225 msec). The reaction times (RTs) for echoing unimodal auditory stimuli were approximately 100 msec faster than the RTs for naming their visual forms. Correspondingly, bimodal facilitation violated race model predictions, but only at SOA values greater than 75 msec. These results indicate that the acoustic and visual information are pooled prior to verbal response programming. However, full expression of this bimodal summation is dependent on the central coincidence of the visual and auditory inputs. These results are considered in the context of studies demonstrating multimodal activation of regions involved in speech production.

Inhibitory processes mediate saccadic target selection.

Observers (N = 3, all males, M age = 30.7 yr.) were presented pairs of visual targets and instructed to fixate one as quickly as possible. The resulting saccades were of two types; bistable saccades accurately acquired one of the targets whereas averaging saccades landed at intermediate locations. The hypothesis was that averaging saccades result from a pooling of afferent activity prior to selection of saccadic direction and amplitude (i.e., coactive parallel processing) and that bistable saccades would show evidence of response competition between incompatible motor programs within a parallel architecture. Analysis of the distributions of saccadic reaction times showed that in all cases saccadic responses to double targets were too slow to be consistent with either the channel summation or any form of race model. Results indicate that inhibitory interactions operate during the selection of a target for a saccadic eye movement.

Directional uncertainty in visually guided pointing.

Studies of the relationship between stimulus-response uncertainty and reaction times indicate three qualitatively different functions: Hick's law, simple-choice step function, or flat curve (no effect at all). The extent of stimulus-response S-R) compatibility appears to interact with the effects of uncertainty on response times. One possible hypothesis regarding these various S-R uncertainty functions is that uncertainty will have an effect whenever the stimuli and their associated responses are not within the same egocentric spatial coordinates. We tested this hypothesis in 5 undergraduate participants (2 men, M age 18.7 yr., range 18-20) by investigating the time-course of pointing to peripherally located visual targets under four different levels of uncertainty (1, 2, 4, or 8 possible locations). Surprisingly, the resulting response function does not match any of those previously reported. Visually guided pointing produced a quadratic reaction time function as S-R uncertainty increases in log2 steps from 1 to 8.

Smooth pursuit of nonvisual motion.

Unlike saccades, smooth pursuit eye movements (SPEMs) are not under voluntary control and their initiation generally requires a moving visual target. However, there are various reports of limited smooth pursuit of the motion of a subject's own finger in total darkness (pursuit based on proprioceptive feedback) and to the combination of proprioception and tactile motion as an unseen finger was moved voluntarily over a smooth surface. In contrast, SPEMs to auditory motion are not distinguishable from pursuit of imagined motion. These reports of smooth pursuit of nonvisual motion cues used a variety of paradigms and different stimuli. In addition, the results have often relied primarily on qualitative descriptions of the smooth pursuit. Here, we directly compare measurements of smooth pursuit gain (eye velocity/stimulus velocity) to visual, auditory, proprioceptive, tactile, and combined tactile + proprioceptive motion stimuli. The results demonstrate high gains for visual pursuit, low gains for auditory pursuit, and intermediate, statistically indistinguishable gains for tactile, proprioceptive, and proprioceptive + tactile pursuit.

Effects of directional uncertainty on visually-guided joystick pointing.

Reaction times generally follow the predictions of Hick's law as stimulus-response uncertainty increases, although notable exceptions include the oculomotor system. Saccadic and smooth pursuit eye movement reaction times are independent of stimulus-response uncertainty. Previous research showed that joystick pointing to targets, a motor analog of saccadic eye movements, is only modestly affected by increased stimulus-response uncertainty; however, a no-uncertainty condition (simple reaction time to 1 possible target) was not included. Here, we re-evaluate manual joystick pointing including a no-uncertainty condition. Analysis indicated simple joystick pointing reaction times were significantly faster than choice reaction times. Choice reaction times (2, 4, or 8 possible target locations) only slightly increased as the number of possible targets increased. These data suggest that, as with joystick tracking (a motor analog of smooth pursuit eye movements), joystick pointing is more closely approximated by a simple/choice step function than the log function predicted by Hick's law.

Effects of stimulus-response uncertainty on saccades to near-threshold targets.

Kveraga et al. (2002, Exp Brain Res 146(3):307-14) reported that saccade latencies are immune to the effects of stimulus-response uncertainty and constitute one of the few response systems that violate Hick's law. Similar effects have been reported for keypresses triggered by vibrations of the fingertips, but robust uncertainty effects were subsequently revealed using weak, low-frequency vibrations (Ten Hoopen et al. 1982, Acta Psychol 50:143-157). We wondered whether immunity of saccadic responses would demonstrate a similar intensity-dependency and therefore re-examined the effects of response entropy on saccade latencies using near-threshold visual stimuli. Saccadic latencies remained independent of stimulus-response uncertainty, indicating that saccadic motor programming is unaffected by the duration of the target detection process.

Ocular tracking as a measure of auditory motion perception.

Motion is a potent sub-modality of vision. Motion cues alone can be used to segment images into figure and ground and break camouflage. Specific patterns of motion support vivid percepts of form, guide locomotion by specifying directional heading and the passage of objects, and in case of an impending collision, the time to impact. Visual motion also drives smooth pursuit eye movements (SPEMs) that serve to stabilize the retinal image of objects in motion. In contrast, the auditory system does not appear to be particularly sensitive to motion. We review the ambiguous status of auditory motion processing from the psychophysical and electrophysiological perspectives. We then report the results of two experiments that use ocular tracking performance as an objective measure of the perception of auditory motion in humans. We examine ocular tracking of auditory motion, visual motion, combined auditory + visual motion and imagined motion in both the frontal plane and in depth. The results demonstrate that ocular tracking of auditory motion is no better than ocular tracking of imagined motion. These results are consistent with the suggestion that, unlike the visual system, the human auditory system is not endowed with low-level motion sensitive elements. We hypothesize however, that auditory information may gain access to a recently described high-level motion processing system that is heavily dependent on 'top-down' influences, including attention.

Smooth pursuit under stimulus-response uncertainty.

Simple reaction times (RTs) are typically faster than choice reaction times and increase with uncertainty according to Hick's law. Here we show that smooth pursuit eye movement RTs show no effect of SR uncertainty while joystick tracking shows a step change between SRT and CRT, but no significant increases beyond two choices. The results suggest there is a benefit to pre-programming joystick tracking but not for smooth pursuit eye movements (SPEMs).

Saccades operate in violation of Hick's law.

Hick's law states that response times (RTs) increase in proportion to the logarithm of the number of potential stimulus-response (S-R) alternatives. We hypothesized that time-consuming processes associated with response selection contribute significantly to this effect. We also hypothesized that the latency of saccades might not conform to Hick's law since visually guided saccades can be automatically selected using topographically organized pathways that convert spatially coded visual activity into spatially coded motor commands. We evaluated these hypotheses by examining three response modalities for their compliance with Hick's law: saccades directed to a visual target (prosaccades), saccades directed away from the target (antisaccades) and manual responses in which each digit was associated with a specific target location (key-press responses). Both antisaccades and key-press responses conformed to Hick's law but saccade latencies were completely unaffected by S-R uncertainty. The significance of these findings is considered in terms of the processes of response selection and premotor programming.