Time Is of the Essence: Neural Codes, Synchronies, Oscillations, Architectures.

Time is of the essence in how neural codes, synchronies, and oscillations might function in encoding, representation, transmission, integration, storage, and retrieval of information in brains. This Hypothesis and Theory article examines observed and possible relations between codes, synchronies, oscillations, and types of neural networks they require. Toward reverse-engineering informational functions in brains, prospective, alternative neural architectures incorporating principles from radio modulation and demodulation, active reverberant circuits, distributed content-addressable memory, signal-signal time-domain correlation and convolution operations, spike-correlation-based holography, and self-organizing, autoencoding anticipatory systems are outlined. Synchronies and oscillations are thought to subserve many possible functions: sensation, perception, action, cognition, motivation, affect, memory, attention, anticipation, and imagination. These include direct involvement in coding attributes of events and objects through phase-locking as well as characteristic patterns of spike latency and oscillatory response. They are thought to be involved in segmentation and binding, working memory, attention, gating and routing of signals, temporal reset mechanisms, inter-regional coordination, time discretization, time-warping transformations, and support for temporal wave-interference based operations. A high level, partial taxonomy of neural codes consists of channel, temporal pattern, and spike latency codes. The functional roles of synchronies and oscillations in candidate neural codes, including oscillatory phase-offset codes, are outlined. Various forms of multiplexing neural signals are considered: time-division, frequency-division, code-division, oscillatory-phase, synchronized channels, oscillatory hierarchies, polychronous ensembles. An expandable, annotative neural spike train framework for encoding low- and high-level attributes of events and objects is proposed. Coding schemes require appropriate neural architectures for their interpretation. Time-delay, oscillatory, wave-interference, synfire chain, polychronous, and neural timing networks are discussed. Some novel concepts for formulating an alternative, more time-centric theory of brain function are discussed. As in radio communication systems, brains can be regarded as networks of dynamic, adaptive transceivers that broadcast and selectively receive multiplexed temporally-patterned pulse signals. These signals enable complex signal interactions that select, reinforce, and bind common subpatterns and create emergent lower dimensional signals that propagate through spreading activation interference networks. If memory traces share the same kind of temporal pattern forms as do active neuronal representations, then distributed, holograph-like content-addressable memories are made possible via temporal pattern resonances.

Noise edge pitch and models of pitch perception.

Monaural noise edge pitch (NEP) is evoked by a broadband noise with a sharp falling edge in the power spectrum. The pitch is heard near the spectral edge frequency but shifted slightly into the frequency region of the noise. Thus, the pitch of a lowpass (LP) noise is matched by a pure tone typically 2%-5% below the edge, whereas the pitch of highpass (HP) noise is matched a comparable amount above the edge. Musically trained listeners can recognize musical intervals between NEPs. The pitches can be understood from a temporal pattern-matching model of pitch perception based on the peaks of a simplified autocorrelation function. The pitch shifts arise from limits on the autocorrelation window duration. An alternative place-theory approach explains the pitch shifts as the result of lateral inhibition. Psychophysical experiments using edge frequencies of 100 Hz and below find that LP-noise pitches exist but HP-noise pitches do not. The result is consistent with a temporal analysis in tonotopic regions outside the noise band. LP and HP experiments with high-frequency edges find that pitch tends to disappear as the edge frequency approaches 5000 Hz, as expected from a timing theory, though exceptional listeners can go an octave higher.

A roadmap for the study of conscious audition and its neural basis.

How and which aspects of neural activity give rise to subjective perceptual experience-i.e. conscious perception-is a fundamental question of neuroscience. To date, the vast majority of work concerning this question has come from vision, raising the issue of generalizability of prominent resulting theories. However, recent work has begun to shed light on the neural processes subserving conscious perception in other modalities, particularly audition. Here, we outline a roadmap for the future study of conscious auditory perception and its neural basis, paying particular attention to how conscious perception emerges (and of which elements or groups of elements) in complex auditory scenes. We begin by discussing the functional role of the auditory system, particularly as it pertains to conscious perception. Next, we ask: what are the phenomena that need to be explained by a theory of conscious auditory perception? After surveying the available literature for candidate neural correlates, we end by considering the implications that such results have for a general theory of conscious perception as well as prominent outstanding questions and what approaches/techniques can best be used to address them.This article is part of the themed issue 'Auditory and visual scene analysis'.

Synchrony capture filterbank: auditory-inspired signal processing for tracking individual frequency components in speech.

A processing scheme for speech signals is proposed that emulates synchrony capture in the auditory nerve. The role of stimulus-locked spike timing is important for representation of stimulus periodicity, low frequency spectrum, and spatial location. In synchrony capture, dominant single frequency components in each frequency region impress their time structures on temporal firing patterns of auditory nerve fibers with nearby characteristic frequencies (CFs). At low frequencies, for voiced sounds, synchrony capture divides the nerve into discrete CF territories associated with individual harmonics. An adaptive, synchrony capture filterbank (SCFB) consisting of a fixed array of traditional, passive linear (gammatone) filters cascaded with a bank of adaptively tunable, bandpass filter triplets is proposed. Differences in triplet output envelopes steer triplet center frequencies via voltage controlled oscillators (VCOs). The SCFB exhibits some cochlea-like responses, such as two-tone suppression and distortion products, and possesses many desirable properties for processing speech, music, and natural sounds. Strong signal components dominate relatively greater numbers of filter channels, thereby yielding robust encodings of relative component intensities. The VCOs precisely lock onto harmonics most important for formant tracking, pitch perception, and sound separation.

Persistent restoration of sensory function by immediate or delayed systemic artemin after dorsal root injury.

Dorsal root injury results in substantial and often irreversible loss of sensory functions as a result of the limited regenerative capacity of sensory axons and the inhibitory barriers that prevent both axonal entry into and regeneration in the spinal cord. Here, we describe previously unknown effects of the growth factor artemin after crush injury of the dorsal spinal nerve roots in rats. Artemin not only promoted re-entry of multiple classes of sensory fibers into the spinal cord and re-establishment of synaptic function and simple behavior, but it also, surprisingly, promoted the recovery of complex behavior. These effects occurred after a 2-week schedule of intermittent, systemic administration of artemin and persisted for at least 6 months following treatment, suggesting a substantial translational advantage. Systemic artemin administration produced essentially complete and persistent restoration of nociceptive and sensorimotor functions, and could represent a promising therapy that may effectively promote sensory neuronal regeneration and functional recovery after injury.

Encoding of pitch in the human brainstem is sensitive to language experience.

Neural processes underlying pitch perception at the level of the cerebral cortex are influenced by language experience. We investigated whether early, pre-attentive stages of pitch processing at the level of the human brainstem may also be influenced by language experience. The human frequency following response (FFR), reflecting sustained phase-locked activity in a population of neural elements, was used to measure activity within the rostral brainstem. FFRs elicited by four Mandarin tones were recorded from native speakers of Mandarin Chinese and English. Pitch strength (reflecting robustness of neural phase-locking at the pitch periods) and accuracy of pitch tracking were extracted from the FFRs using autocorrelation algorithms. These measures revealed that the Chinese group exhibits stronger pitch representation and smoother pitch tracking than the English group. Consistent with the pitch data, FFR spectral data showed that the Chinese group exhibits stronger representation of the second harmonic relative to the English group across all four tones. These results cannot be explained by a temporal pitch encoding scheme which simply extracts the dominant interspike interval. Rather, these results support the possibility of neural plasticity at the brainstem level that is induced by language experience that may be enhancing or priming linguistically relevant features of the speech input.

Neurophysiology and neuroanatomy of pitch perception: auditory cortex.

We present original results and review literature from the past fifty years that address the role of primate auditory cortex in the following perceptual capacities: (1) the ability to perceive small differences between the pitches of two successive tones; (2) the ability to perceive the sign (i.e., direction) of the pitch difference [higher (+) vs. lower (-)]; and (3) the ability to abstract pitch constancy across changes in stimulus acoustics. Cortical mechanisms mediating pitch perception are discussed with respect to (1) gross and microanatomical distribution; and (2) candidate neural coding schemes. Observations by us and others suggest that (1) frequency-selective neurons in primary auditory cortex (A1) and surrounding fields play a critical role in fine-grained pitch discrimination at the perceptual level; (2) cortical mechanisms that detect pitch differences are neuroanatomically dissociable from those mediating pitch direction discrimination; (3) cortical mechanisms mediating perception of the "missing fundamental frequency (F0)" are neuroanatomically dissociable from those mediating pitch perception when F0 is present; (4) frequency-selective neurons in both right and left A1 contribute to pitch change detection and pitch direction discrimination; (5) frequency-selective neurons in right A1 are necessary for normal pitch direction discrimination; (6) simple codes for pitch that are based on single- and multiunit firing rates of frequency-selective neurons face both a "hyperacuity problem" and a "pitch constancy problem"-that is, frequency discrimination thresholds for pitch change direction and pitch direction discrimination are much smaller than neural tuning curves predict, and firing rate patterns change dramatically under conditions in which pitch percepts remain invariant; (7) cochleotopic organization of frequency-selective neurons bears little if any relevance to perceptual acuity and pitch constancy; and (8) simple temporal codes for pitch capable of accounting for pitches higher than a few hundred hertz have not been found in the auditory cortex. The cortical code for pitch is therefore not likely to be a function of simple rate profiles or synchronous temporal patterns. Studies motivated by interest in the neurophysiology and neuroanatomy of music perception have helped correct longstanding misconceptions about the functional role of auditory cortex in frequency discrimination and pitch perception. Advancing knowledge about the neural coding of pitch is of fundamental importance to the future design of neurobionic therapies for hearing loss.

Temporal codes and computations for sensory representation and scene analysis.

This paper considers a space of possible temporal codes, surveys neurophysiological and psychological evidence for their use in nervous systems, and presents examples of neural timing networks that operate in the time-domain. Sensory qualities can be encoded temporally by means of two broad strategies: stimulus-driven temporal correlations (phase-locking) and stimulus-triggering of endogenous temporal response patterns. Evidence for stimulus-related spike timing patterns exists in nearly every sensory modality, and such information can be potentially utilized for representation of stimulus qualities, localization of sources, and perceptual grouping. Multiple strategies for temporal (time, frequency, and code-division) multiplexing of information for transmission and grouping are outlined. Using delays and multiplications (coincidences), neural timing networks perform time-domain signal processing operations to compare, extract and separate temporal patterns. Separation of synthetic double vowels by a recurrent neural timing network is used to illustrate how coherences in temporal fine structure can be exploited to build up and separate periodic signals with different fundamentals. Timing nets constitute a time-domain scene analysis strategy based on temporal pattern invariance rather than feature-based labeling, segregation and binding of channels. Further potential implications of temporal codes and computations for new kinds of neural networks are explored.

Human frequency-following response: representation of pitch contours in Chinese tones.

Auditory nerve single-unit population studies have demonstrated that phase-locking plays a dominant role in the neural encoding of both the spectrum and voice pitch of speech sounds. Phase-locked neural activity underlying the scalp-recorded human frequency-following response (FFR) has also been shown to encode certain spectral features of steady-state and time-variant speech sounds as well as pitch of several complex sounds that produce time-invariant pitch percepts. By extension, it was hypothesized that the human FFR may preserve pitch-relevant information for speech sounds that elicit time-variant as well as steady-state pitch percepts. FFRs were elicited in response to the four lexical tones of Mandarin Chinese as well as to a complex auditory stimulus which was spectrally different but equivalent in fundamental frequency (f0) contour to one of the Chinese tones. Autocorrelation-based pitch extraction measures revealed that the FFR does indeed preserve pitch-relevant information for all stimuli. Phase-locked interpeak intervals closely followed f0. Spectrally different stimuli that were equivalent in F0 similarly showed robust interpeak intervals that followed f0. These FFR findings support the viability of early, population-based 'predominant interval' representations of pitch in the auditory brainstem that are based on temporal patterns of phase-locked neural activity.

Extradimensional bypass.

We discuss the concept of extradimensional bypass as it was developed by the late theoretical biologist Michael Conrad. An evolving system that optimizes its performance by gradient ascent (hill climbing) can avoid being trapped in local maxima by increasing the effective dimensionality of its search space. Many local maxima may become saddle points in the higher dimensional space, such that gradient ascent can continue unimpeded. Extradimensional bypass as a concept has parallels in theories of open-ended learning and functional emergence, where new structural, functional, and informational primitives can increase the effective dimensionality of material systems.