Body-part specificity for learning of multiple prior distributions in human coincidence timing.

During timing tasks, the brain learns the statistical distribution of target intervals and integrates this prior knowledge with sensory inputs to optimise task performance. Daily events can have different temporal statistics (e.g., fastball/slowball in baseball batting), making it important to learn and retain multiple priors. However, the rules governing this process are not yet understood. Here, we demonstrate that the learning of multiple prior distributions in a coincidence timing task is characterised by body-part specificity. In our experiments, two prior distributions (short and long intervals) were imposed on participants. When using only one body part for timing responses, regardless of the priors, participants learned a single prior by generalising over the two distributions. However, when the two priors were assigned to different body parts, participants concurrently learned the two independent priors. Moreover, body-part specific prior acquisition was faster when the priors were assigned to anatomically distant body parts (e.g., hand/foot) than when they were assigned to close body parts (e.g., index/middle fingers). This suggests that the body-part specific learning of priors is organised according to somatotopy.

Publisher Correction: Temporal rate is not a distinct perceptual metric.

An amendment to this paper has been published and can be accessed via a link at the top of the paper.

Temporal rate is not a distinct perceptual metric.

Sensory adaptation experiments have revealed the existence of 'rate after-effects' - adapting to a relatively fast rate makes an intermediate test rate feel slow, and adapting to a slow rate makes the same moderate test rate feel fast. The present work aims to deconstruct the concept of rate and clarify how exactly the brain processes a regular sequence of sensory signals. We ask whether rate forms a distinct perceptual metric, or whether it is simply the perceptual aggregate of the intervals between its component signals. Subjects were exposed to auditory or visual temporal rates (a 'slow' rate of 1.5 Hz and a 'fast' rate of 6 Hz), before being tested with single unfilled intervals of varying durations. Results show adapting to a given rate strongly influences the perceived duration of a single empty interval. This effect is robust across both interval reproduction and duration discrimination judgments. These findings challenge our understanding of rate perception. Specifically, they suggest that contrary to some previous assertions, the perception of sequence rate is strongly influenced by the perception of the sequence's component duration intervals.

Adaptation reveals multi-stage coding of visual duration.

In conflict with historically dominant models of time perception, recent evidence suggests that the encoding of our environment's temporal properties may not require a separate class of neurons whose raison d'être is the dedicated processing of temporal information. If true, it follows that temporal processing should be imbued with the known selectivity found within non-temporal neurons. In the current study, we tested this hypothesis for the processing of a poorly understood stimulus parameter: visual event duration. We used sensory adaptation techniques to generate duration aftereffects: bidirectional distortions of perceived duration. Presenting adapting and test durations to the same vs different eyes utilises the visual system's anatomical progression from monocular, pre-cortical neurons to their binocular, cortical counterparts. Duration aftereffects exhibited robust inter-ocular transfer alongside a small but significant contribution from monocular mechanisms. We then used novel stimuli which provided duration information that was invisible to monocular neurons. These stimuli generated robust duration aftereffects which showed partial selectivity for adapt-test changes in retinal disparity. Our findings reveal distinct duration encoding mechanisms at monocular, depth-selective and depth-invariant stages of the visual hierarchy.

Rate after-effects fail to transfer cross-modally: Evidence for distributed sensory timing mechanisms.

Accurate time perception is critical for a number of human behaviours, such as understanding speech and the appreciation of music. However, it remains unresolved whether sensory time perception is mediated by a central timing component regulating all senses, or by a set of distributed mechanisms, each dedicated to a single sensory modality and operating in a largely independent manner. To address this issue, we conducted a range of unimodal and cross-modal rate adaptation experiments, in order to establish the degree of specificity of classical after-effects of sensory adaptation. Adapting to a fast rate of sensory stimulation typically makes a moderate rate appear slower (repulsive after-effect), and vice versa. A central timing hypothesis predicts general transfer of adaptation effects across modalities, whilst distributed mechanisms predict a high degree of sensory selectivity. Rate perception was quantified by a method of temporal reproduction across all combinations of visual, auditory and tactile senses. Robust repulsive after-effects were observed in all unimodal rate conditions, but were not observed for any cross-modal pairings. Our results show that sensory timing abilities are adaptable but, crucially, that this change is modality-specific - an outcome that is consistent with a distributed sensory timing hypothesis.

Generalization of prior information for rapid Bayesian time estimation.

To enable effective interaction with the environment, the brain combines noisy sensory information with expectations based on prior experience. There is ample evidence showing that humans can learn statistical regularities in sensory input and exploit this knowledge to improve perceptual decisions and actions. However, fundamental questions remain regarding how priors are learned and how they generalize to different sensory and behavioral contexts. In principle, maintaining a large set of highly specific priors may be inefficient and restrict the speed at which expectations can be formed and updated in response to changes in the environment. However, priors formed by generalizing across varying contexts may not be accurate. Here, we exploit rapidly induced contextual biases in duration reproduction to reveal how these competing demands are resolved during the early stages of prior acquisition. We show that observers initially form a single prior by generalizing across duration distributions coupled with distinct sensory signals. In contrast, they form multiple priors if distributions are coupled with distinct motor outputs. Together, our findings suggest that rapid prior acquisition is facilitated by generalization across experiences of different sensory inputs but organized according to how that sensory information is acted on.

Object size determines the spatial spread of visual time.

A key question for temporal processing research is how the nervous system extracts event duration, despite a notable lack of neural structures dedicated to duration encoding. This is in stark contrast with the orderly arrangement of neurons tasked with spatial processing. In this study, we examine the linkage between the spatial and temporal domains. We use sensory adaptation techniques to generate after-effects where perceived duration is either compressed or expanded in the opposite direction to the adapting stimulus' duration. Our results indicate that these after-effects are broadly tuned, extending over an area approximately five times the size of the stimulus. This region is directly related to the size of the adapting stimulus-the larger the adapting stimulus the greater the spatial spread of the after-effect. We construct a simple model to test predictions based on overlapping adapted versus non-adapted neuronal populations and show that our effects cannot be explained by any single, fixed-scale neural filtering. Rather, our effects are best explained by a self-scaled mechanism underpinned by duration selective neurons that also pool spatial information across earlier stages of visual processing.

A neural hierarchy for illusions of time: duration adaptation precedes multisensory integration.

Perceived time is inherently malleable. For example, adaptation to relatively long or short sensory events leads to a repulsive aftereffect such that subsequent events appear to be contracted or expanded (duration adaptation). Perceived visual duration can also be distorted via concurrent presentation of discrepant auditory durations (multisensory integration). The neural loci of both distortions remain unknown. In the current study we use a psychophysical approach to establish their relative positioning within the sensory processing hierarchy. We show that audiovisual integration induces marked distortions of perceived visual duration. We proceed to use these distorted durations as visual adapting stimuli yet find subsequent visual duration aftereffects to be consistent with physical rather than perceived visual duration. Conversely, the concurrent presentation of adapted auditory durations with nonadapted visual durations results in multisensory integration patterns consistent with perceived, rather than physical, auditory duration. These results demonstrate that recent sensory history modifies human duration perception prior to the combination of temporal information across sensory modalities and provides support for adaptation mechanisms mediated by duration selective neurons situated in early areas of the visual and auditory nervous system (Aubie, Sayegh, & Faure, 2012; Duysens, Schaafsma, & Orban, 1996; Leary, Edwards, & Rose, 2008).

Audiovisual time perception is spatially specific.

Our sensory systems face a daily barrage of auditory and visual signals whose arrival times form a wide range of audiovisual asynchronies. These temporal relationships constitute an important metric for the nervous system when surmising which signals originate from common external events. Internal consistency is known to be aided by sensory adaptation: repeated exposure to consistent asynchrony brings perceived arrival times closer to simultaneity. However, given the diverse nature of our audiovisual environment, functionally useful adaptation would need to be constrained to signals that were generated together. In the current study, we investigate the role of two potential constraining factors: spatial and contextual correspondence. By employing an experimental design that allows independent control of both factors, we show that observers are able to simultaneously adapt to two opposing temporal relationships, provided they are segregated in space. No such recalibration was observed when spatial segregation was replaced by contextual stimulus features (in this case, pitch and spatial frequency). These effects provide support for dedicated asynchrony mechanisms that interact with spatially selective mechanisms early in visual and auditory sensory pathways.

Duration channels mediate human time perception.

The task of deciding how long sensory events seem to last is one that the human nervous system appears to perform rapidly and, for sub-second intervals, seemingly without conscious effort. That these estimates can be performed within and between multiple sensory and motor domains suggest time perception forms one of the core, fundamental processes of our perception of the world around us. Given this significance, the current paucity in our understanding of how this process operates is surprising. One candidate mechanism for duration perception posits that duration may be mediated via a system of duration-selective 'channels', which are differentially activated depending on the match between afferent duration information and the channels' 'preferred' duration. However, this model awaits experimental validation. In the current study, we use the technique of sensory adaptation, and we present data that are well described by banks of duration channels that are limited in their bandwidth, sensory-specific, and appear to operate at a relatively early stage of visual and auditory sensory processing. Our results suggest that many of the computational principles the nervous system applies to coding visual spatial and auditory spectral information are common to its processing of temporal extent.

Asynchrony adaptation reveals neural population code for audio-visual timing.

The relative timing of auditory and visual stimuli is a critical cue for determining whether sensory signals relate to a common source and for making inferences about causality. However, the way in which the brain represents temporal relationships remains poorly understood. Recent studies indicate that our perception of multisensory timing is flexible--adaptation to a regular inter-modal delay alters the point at which subsequent stimuli are judged to be simultaneous. Here, we measure the effect of audio-visual asynchrony adaptation on the perception of a wide range of sub-second temporal relationships. We find distinctive patterns of induced biases that are inconsistent with the previous explanations based on changes in perceptual latency. Instead, our results can be well accounted for by a neural population coding model in which: (i) relative audio-visual timing is represented by the distributed activity across a relatively small number of neurons tuned to different delays; (ii) the algorithm for reading out this population code is efficient, but subject to biases owing to under-sampling; and (iii) the effect of adaptation is to modify neuronal response gain. These results suggest that multisensory timing information is represented by a dedicated population code and that shifts in perceived simultaneity following asynchrony adaptation arise from analogous neural processes to well-known perceptual after-effects.

Attention regulates the plasticity of multisensory timing.

Evidence suggests than human time perception is likely to reflect an ensemble of recent temporal experience. For example, prolonged exposure to consistent temporal patterns can adaptively realign the perception of event order, both within and between sensory modalities (e.g. Fujisaki et al., 2004 Nat. Neurosci., 7, 773-778). In addition, the observation that 'a watched pot never boils' serves to illustrate the fact that dynamic shifts in our attentional state can also produce marked distortions in our temporal estimates. In the current study we provide evidence for a hitherto unknown link between adaptation, temporal perception and our attentional state. We show that our ability to use recent sensory history as a perceptual baseline for ongoing temporal judgments is subject to striking top-down modulation via shifts in the observer's selective attention. Specifically, attending to the temporal structure of asynchronous auditory and visual adapting stimuli generates a substantial increase in the temporal recalibration induced by these stimuli. We propose a conceptual framework accounting for our findings whereby attention modulates the perceived salience of temporal patterns. This heightened salience allows the formation of audiovisual perceptual 'objects', defined solely by their temporal structure. Repeated exposure to these objects induces high-level pattern adaptation effects, akin to those found in visual and auditory domains (e.g. Leopold & Bondar (2005) Fitting the Mind to the World: Adaptation and Aftereffects in High-Level Vision. Oxford University Press, Oxford, 189-211; Schweinberger et al. (2008) Curr. Biol., 18, 684-688).

Effect before cause: supramodal recalibration of sensorimotor timing.

BACKGROUND: Our motor actions normally generate sensory events, but how do we know which events were self generated and which have external causes? Here we use temporal adaptation to investigate the processing stage and generality of our sensorimotor timing estimates. METHODOLOGY/PRINCIPAL FINDINGS: Adaptation to artificially-induced delays between action and event can produce a startling percept--upon removal of the delay it feels as if the sensory event precedes its causative action. This temporal recalibration of action and event occurs in a quantitatively similar manner across the sensory modalities. Critically, it is robust to the replacement of one sense during the adaptation phase with another sense during the test judgment. CONCLUSIONS/SIGNIFICANCE: Our findings suggest a high-level, supramodal recalibration mechanism. The effects are well described by a simple model which attempts to preserve the expected synchrony between action and event, but only when causality indicates it is reasonable to do so. We further demonstrate that this model successfully characterises related adaptation data from outside the sensorimotor domain.

Preferential processing of tactile events under conditions of divided attention.

Differences in transduction and transmission latencies of visual, auditory and tactile events cause corresponding differences in simple reaction time. As reaction time is usually measured in unimodal blocks, it is unclear whether such latency differences also apply when observers monitor multiple sensory channels. We investigate this by comparing reaction time when attention is focused on a single modality, and when attention is divided between multiple modalities. Results show that tactile reaction time is unaffected by dividing attention, whereas visual and auditory reaction times are significantly and asymmetrically increased. These findings show that tactile information is processed preferentially by the nervous system under conditions of divided attention, and suggest that tactile events may be processed preattentively.

Recalibration of perceived time across sensory modalities.

When formulating an estimate of event time, the human sensory system has been shown to possess a degree of perceptual flexibility. Specifically, the perceived relative timing of auditory and visual stimuli is, to some extent, a product of recent experience. It has been suggested that this form of sensory recalibration may be peculiar to the audiovisual domain. Here we investigate how adaptation to sensory asynchrony influences the perceived temporal order of audiovisual, audiotactile and visuotactile stimulus pairs. Our data show that a brief period of repeated exposure to asynchrony in any of these sensory pairings results in marked changes in subsequent temporal order judgments: the point of perceived simultaneity shifts toward the level of adaptation asynchrony. We find that the size and nature of this shift is very similar in all three pairings and that sensitivity to asynchrony is unaffected by the adaptation process. In light of these findings we suggest that a single supramodal mechanism may be responsible for the observed recalibration of multisensory perceived time.

Adaptation minimizes distance-related audiovisual delays.

A controversial hypothesis within the domain of sensory research is that observers are able to use visual and auditory distance cues to maintain perceptual synchrony--despite the differential velocities of light and sound. Here we show that observers are categorically unable to utilize such distance cues. Nevertheless, given a period of adaptation to the naturally occurring audiovisual asynchrony associated with each viewing distance, a temporal recalibration mechanism helps to perceptually compensate for the effects of distance-induced auditory delays. These effects demonstrate a novel functionality of temporal recalibration with clear ecological benefits.

Resolving multisensory conflict: a strategy for balancing the costs and benefits of audio-visual integration.

In order to maintain a coherent, unified percept of the external environment, the brain must continuously combine information encoded by our different sensory systems. Contemporary models suggest that multisensory integration produces a weighted average of sensory estimates, where the contribution of each system to the ultimate multisensory percept is governed by the relative reliability of the information it provides (maximum-likelihood estimation). In the present study, we investigate interactions between auditory and visual rate perception, where observers are required to make judgments in one modality while ignoring conflicting rate information presented in the other. We show a gradual transition between partial cue integration and complete cue segregation with increasing inter-modal discrepancy that is inconsistent with mandatory implementation of maximum-likelihood estimation. To explain these findings, we implement a simple Bayesian model of integration that is also able to predict observer performance with novel stimuli. The model assumes that the brain takes into account prior knowledge about the correspondence between auditory and visual rate signals, when determining the degree of integration to implement. This provides a strategy for balancing the benefits accrued by integrating sensory estimates arising from a common source, against the costs of conflating information relating to independent objects or events.