Temporal Coherence Shapes Cortical Responses to Speech Mixtures in a Ferret Cocktail Party.

Segregation of complex sounds such as speech, music and animal vocalizations as they simultaneously emanate from multiple sources (referred to as the "cocktail party problem") is a remarkable ability that is common in humans and animals alike. The neural underpinnings of this process have been extensively studied behaviorally and physiologically in non-human animals primarily with simplified sounds (tones and noise sequences). In humans, segregation experiments utilizing more complex speech mixtures are common; but physiological experiments have relied on EEG/MEG/ECoG recordings that sample activity from thousands of neurons, often obscuring the detailed processes that give rise to the observed segregation. The present study combines the insights from animal single-unit physiology with segregation of speech-like mixtures. Ferrets were trained to attend to a female voice and detect a target word, both in presence or absence of a concurrent, equally salient male voice. Single neuron recordings were obtained from primary and secondary ferret auditory cortical fields, as well as frontal cortex. During task performance, representation of the female words became more enhanced relative to those of the (distractor) male in all cortical regions, especially in the higher auditory cortical field. Analysis of the temporal and spectral response characteristics during task performance reveals how speech segregation gradually emerges in the auditory cortex. A computational model evaluated on the same voice mixtures replicates and extends these results to different attentional targets (attention to female or male voices). These findings are consistent with the temporal coherence theory whereby attention to a target voice anchors neural activity in cortical networks hence binding together channels that are coherently temporally-modulated with the target, and ultimately forming a common auditory stream.

Dorsal prefrontal and premotor cortex of the ferret as defined by distinctive patterns of thalamo-cortical projections.

Recent studies of the neurobiology of the dorsal frontal cortex (FC) of the ferret have illuminated its key role in the attention network, top-down cognitive control of sensory processing, and goal directed behavior. To elucidate the neuroanatomical regions of the dorsal FC, and delineate the boundary between premotor cortex (PMC) and dorsal prefrontal cortex (dPFC), we placed retrograde tracers in adult ferret dorsal FC anterior to primary motor cortex and analyzed thalamo-cortical connectivity. Cyto- and myeloarchitectural differences across dorsal FC and the distinctive projection patterns from thalamic nuclei, especially from the subnuclei of the medial dorsal (MD) nucleus and the ventral thalamic nuclear group, make it possible to clearly differentiate three separate dorsal FC fields anterior to primary motor cortex: polar dPFC (dPFCpol), dPFC, and PMC. Based on the thalamic connectivity, there is a striking similarity of the ferret's dorsal FC fields with other species. This possible homology opens up new questions for future comparative neuroanatomical and functional studies.

Dynamics and Hierarchical Encoding of Non-compact Acoustic Categories in Auditory and Frontal Cortex.

Categorical perception is a fundamental cognitive function enabling animals to flexibly assign sounds into behaviorally relevant categories. This study investigates the nature of acoustic category representations, their emergence in an ascending series of ferret auditory and frontal cortical fields, and the dynamics of this representation during passive listening to task-relevant stimuli and during active retrieval from memory while engaging in learned categorization tasks. Ferrets were trained on two auditory Go-NoGo categorization tasks to discriminate two non-compact sound categories (composed of tones or amplitude-modulated noise). Neuronal responses became progressively more categorical in higher cortical fields, especially during task performance. The dynamics of the categorical responses exhibited a cascading top-down modulation pattern that began earliest in the frontal cortex and subsequently flowed downstream to the secondary auditory cortex, followed by the primary auditory cortex. In a subpopulation of neurons, categorical responses persisted even during the passive listening condition, demonstrating memory for task categories and their enhanced categorical boundaries.

State-dependent encoding of sound and behavioral meaning in a tertiary region of the ferret auditory cortex.

In higher sensory cortices, there is a gradual transformation from sensation to perception and action. In the auditory system, this transformation is revealed by responses in the rostral ventral posterior auditory field (VPr), a tertiary area in the ferret auditory cortex, which shows long-term learning in trained compared to naïve animals, arising from selectively enhanced responses to behaviorally relevant target stimuli. This enhanced representation is further amplified during active performance of spectral or temporal auditory discrimination tasks. VPr also shows sustained short-term memory activity after target stimulus offset, correlated with task response timing and action. These task-related changes in auditory filter properties enable VPr neurons to quickly and nimbly switch between different responses to the same acoustic stimuli, reflecting either spectrotemporal properties, timing, or behavioral meaning of the sound. Furthermore, they demonstrate an interaction between the dynamics of short-term attention and long-term learning, as incoming sound is selectively attended, recognized, and translated into action.

Go/No-Go task engagement enhances population representation of target stimuli in primary auditory cortex.

Primary sensory cortices are classically considered to extract and represent stimulus features, while association and higher-order areas are thought to carry information about stimulus meaning. Here we show that this information can in fact be found in the neuronal population code of the primary auditory cortex (A1). A1 activity was recorded in awake ferrets while they either passively listened or actively discriminated stimuli in a range of Go/No-Go paradigms, with different sounds and reinforcements. Population-level dimensionality reduction techniques reveal that task engagement induces a shift in stimulus encoding from a sensory to a behaviorally driven representation that specifically enhances the target stimulus in all paradigms. This shift partly relies on task-engagement-induced changes in spontaneous activity. Altogether, we show that A1 population activity bears strong similarities to frontal cortex responses. These findings indicate that primary sensory cortices implement a crucial change in the structure of population activity to extract task-relevant information during behavior.

Orbitofrontal Cortex Neurons Respond to Sound and Activate Primary Auditory Cortex Neurons.

Sensory environments change over a wide dynamic range and sensory processing can change rapidly to facilitate stable perception. While rapid changes may occur throughout the sensory processing pathway, cortical changes are believed to profoundly influence perception. Prior stimulation studies showed that orbitofrontal cortex (OFC) can modify receptive fields and sensory coding in A1, but the engagement of OFC during listening and the pathways mediating OFC influences on A1 are unknown. We show in mice that OFC neurons respond to sounds consistent with a role of OFC in audition. We then show in vitro that OFC axons are present in A1 and excite pyramidal and GABAergic cells in all layers of A1 via glutamatergic synapses. Optogenetic stimulation of OFC terminals in A1 in vivo evokes short-latency neural activity in A1 and pairing activation of OFC projections in A1 with sounds alters sound-evoked A1 responses. Together, our results identify a direct connection from OFC to A1 that can excite A1 neurons at the earliest stage of cortical processing, and thereby sculpt A1 receptive fields. These results are consistent with a role for OFC in adjusting to changing behavioral relevance of sensory inputs and modulating A1 receptive fields to enhance sound processing.

Temporal coherence structure rapidly shapes neuronal interactions.

Perception of segregated sources is essential in navigating cluttered acoustic environments. A basic mechanism to implement this process is the temporal coherence principle. It postulates that a signal is perceived as emitted from a single source only when all of its features are temporally modulated coherently, causing them to bind perceptually. Here we report on neural correlates of this process as rapidly reshaped interactions in primary auditory cortex, measured in three different ways: as changes in response rates, as adaptations of spectrotemporal receptive fields following stimulation by temporally coherent and incoherent tone sequences, and as changes in spiking correlations during the tone sequences. Responses, sensitivity and presumed connectivity were rapidly enhanced by synchronous stimuli, and suppressed by alternating (asynchronous) sounds, but only when the animals engaged in task performance and were attentive to the stimuli. Temporal coherence and attention are therefore both important factors in auditory scene analysis.

Relative salience of spectral and temporal features in auditory long-term memory.

In order to explore the representation of sound features in auditory long-term memory, two groups of ferrets were trained on Go vs Nogo, 3-zone classification tasks. The sound stimuli differed primarily along the spectral and temporal dimensions. In Group 1, two ferrets were trained to (i) classify tones based on their frequency (Tone-task), and subsequently learned to (ii) classify white noise based on its amplitude modulation rate (AM-task). In Group 2, two ferrets were trained to classify tones based on correlated combinations of their frequency and AM rate (AM-Tone task). Both groups of ferrets learned their tasks and were able to generalize performance along the trained spectral (tone frequency) or temporal (AM rate) dimensions. Insights into stimulus representations in memory were gained when the animals were tested with a diverse set of untrained probes that mixed features from the two dimensions. Animals exhibited a complex pattern of responses to the probes reflecting primarily the probes' spectral similarity with the training stimuli, and secondarily the temporal features of the stimuli. These diverse behavioral decisions could be well accounted for by a nearest-neighbor classifier model that relied on a multiscale spectrotemporal cortical representation of the training and probe sounds.

Neural correlates of auditory short-term memory in rostral superior temporal cortex.

BACKGROUND: Auditory short-term memory (STM) in the monkey is less robust than visual STM and may depend on a retained sensory trace, which is likely to reside in the higher-order cortical areas of the auditory ventral stream. RESULTS: We recorded from the rostral superior temporal cortex as monkeys performed serial auditory delayed match-to-sample (DMS). A subset of neurons exhibited modulations of their firing rate during the delay between sounds, during the sensory response, or during both. This distributed subpopulation carried a predominantly sensory signal modulated by the mnemonic context of the stimulus. Excitatory and suppressive effects on match responses were dissociable in their timing and in their resistance to sounds intervening between the sample and match. CONCLUSIONS: Like the monkeys' behavioral performance, these neuronal effects differ from those reported in the same species during visual DMS, suggesting different neural mechanisms for retaining dynamic sounds and static images in STM.

Rapid spectrotemporal plasticity in primary auditory cortex during behavior.

Complex natural and environmental sounds, such as speech and music, convey information along both spectral and temporal dimensions. The cortical representation of such stimuli rapidly adapts when animals become actively engaged in discriminating them. In this study, we examine the nature of these changes using simplified spectrotemporal versions (upward vs downward shifting tone sequences) with domestic ferrets (Mustela putorius). Cortical processing rapidly adapted to enhance the contrast between the two discriminated stimulus categories, by changing spectrotemporal receptive field properties to encode both the spectral and temporal structure of the tone sequences. Furthermore, the valence of the changes was closely linked to the task reward structure: stimuli associated with negative reward became enhanced relative to those associated with positive reward. These task- and-stimulus-related spectrotemporal receptive field changes occurred only in trained animals during, and immediately following, behavior. This plasticity was independently confirmed by parallel changes in a directionality function measured from the responses to the transition of tone sequences during task performance. The results demonstrate that induced patterns of rapid plasticity reflect closely the spectrotemporal structure of the task stimuli, thus extending the functional relevance of rapid task-related plasticity to the perception and learning of natural sounds such speech and animal vocalizations.

The basal forebrain modulates spontaneous activity of principal cells in the main olfactory bulb of anesthetized mice.

Spontaneous activity is an important characteristic of the principal cells in the main olfactory bulb (MOB) for encoding odor information, which is modulated by the basal forebrain. Cholinergic activation has been reported to inhibit all major neuron types in the MOB. In this study, the effect of diagonal band (NDB) stimulation on mitral/tufted (M/T) cell spontaneous activity was examined in anesthetized mice. NDB stimulation increased spontaneous activity in 66 MOB neurons which lasted for 2-35 s before returning to the baseline level. The majority of the effected units showed a decrease of interspike intervals (ISI) at a range of 8-25 ms. Fifty-two percent of NDB stimulation responsive units showed intrinsic rhythmical bursting, which was enhanced temporarily by NDB stimulation, whereas the remaining non-rhythmic units were capable of synchronized bursting. The effect was attenuated by scopolamine in 21 of 27 units tested. Only four NDB units were inhibited by NDB stimulation, an inhibition that lasted less than 10 s. The NDB stimulation responsive neurons appeared to be M/T cells. Our findings demonstrate an NDB excitation effect on M/T neurons that mostly requires muscarinic receptor activation, and is likely due to non-selectivity of electrical stimulation. This suggests that cholinergic and a diverse group of non-cholinergic neurons in the basal forebrain co-ordinately modulate the dynamics of M/T cell spontaneous activity, which is fundamental for odor representation and attentional perception.

Temporal coherence and the streaming of complex sounds.

Humans and other animals can attend to one of multiple sounds, and -follow it selectively over time. The neural underpinnings of this perceptual feat remain mysterious. Some studies have concluded that sounds are heard as separate streams when they activate well-separated populations of central auditory neurons, and that this process is largely pre-attentive. Here, we propose instead that stream formation depends primarily on temporal coherence between responses that encode various features of a sound source. Furthermore, we postulate that only when attention is directed toward a particular feature (e.g., pitch or location) do all other temporally coherent features of that source (e.g., timbre and location) become bound together as a stream that is segregated from the incoherent features of other sources. Experimental -neurophysiological evidence in support of this hypothesis will be presented. The focus, however, will be on a computational realization of this idea and a discussion of the insights learned from simulations to disentangle complex sound sources such as speech and music. The model consists of a representational stage of early and cortical auditory processing that creates a multidimensional depiction of various sound attributes such as pitch, location, and spectral resolution. The following stage computes a coherence matrix that summarizes the pair-wise correlations between all channels making up the cortical representation. Finally, the perceived segregated streams are extracted by decomposing the coherence matrix into its uncorrelated components. Questions raised by the model are discussed, especially on the role of attention in streaming and the search for further neural correlates of streaming percepts.

Effect of acoustic similarity on short-term auditory memory in the monkey.

Recent evidence suggests that the monkey's short-term memory in audition depends on a passively retained sensory trace as opposed to a trace reactivated from long-term memory for use in working memory. Reliance on a passive sensory trace could render memory particularly susceptible to confusion between sounds that are similar in some acoustic dimension. If so, then in delayed matching-to-sample, the monkey's performance should be predicted by the similarity in the salient acoustic dimension between the sample and subsequent test stimulus, even at very short delays. To test this prediction and isolate the acoustic features relevant to short-term memory, we examined the pattern of errors made by two rhesus monkeys performing a serial, auditory delayed match-to-sample task with interstimulus intervals of 1 s. The analysis revealed that false-alarm errors did indeed result from similarity-based confusion between the sample and the subsequent nonmatch stimuli. Manipulation of the stimuli showed that removal of spectral cues was more disruptive to matching behavior than removal of temporal cues. In addition, the effect of acoustic similarity on false-alarm response was stronger at the first nonmatch stimulus than at the second one. This pattern of errors would be expected if the first nonmatch stimulus overwrote the sample's trace, and suggests that the passively retained trace is not only vulnerable to similarity-based confusion but is also highly susceptible to overwriting.

Monkeys have a limited form of short-term memory in audition.

A stimulus trace may be temporarily retained either actively [i.e., in working memory (WM)] or by the weaker mnemonic process we will call passive short-term memory, in which a given stimulus trace is highly susceptible to "overwriting" by a subsequent stimulus. It has been suggested that WM is the more robust process because it exploits long-term memory (i.e., a current stimulus activates a stored representation of that stimulus, which can then be actively maintained). Recent studies have suggested that monkeys may be unable to store acoustic signals in long-term memory, raising the possibility that they may therefore also lack auditory WM. To explore this possibility, we tested rhesus monkeys on a serial delayed match-to-sample (DMS) task using a small set of sounds presented with ~1-s interstimulus delays. Performance was accurate whenever a match or a nonmatch stimulus followed the sample directly, but it fell precipitously if a single nonmatch stimulus intervened between sample and match. The steep drop in accuracy was found to be due not to passive decay of the sample's trace, but to retroactive interference from the intervening nonmatch stimulus. This "overwriting" effect was far greater than that observed previously in serial DMS with visual stimuli. The results, which accord with the notion that WM relies on long-term memory, indicate that monkeys perform serial DMS in audition remarkably poorly and that whatever success they had on this task depended largely, if not entirely, on the retention of stimulus traces in the passive form of short-term memory.

A quantitative analysis of information about past and present stimuli encoded by spikes of A1 neurons.

To process the rich temporal structure of their acoustic environment, organisms have to integrate information over time into an appropriate neural response. Previous studies have addressed the modulation of responses of auditory neurons to a current sound in dependence of the immediate stimulation history, but a quantitative analysis of this important computational process has been missing. In this study, we analyzed temporal integration of information in the spike output of 122 single neurons in primary auditory cortex (A1) of four awake ferrets in response to random tone sequences. We quantified the information contained in the responses about both current and preceding sounds in two ways: by estimating directly the mutual information between stimulus and response, and by training linear classifiers to decode information about the stimulus from the neural response. We found that 1) many neurons conveyed a significant amount of information not only about the current tone but also simultaneously about the previous tone, 2) the neural response to tone sequences was a nonlinear combination of responses to the tones in isolation, and 3) nevertheless, much of the information about current and previous tones could be extracted by linear decoders. Furthermore, our analysis of these experimental data shows that methods from information theory and the application of standard machine learning methods for extracting specific information yield quite similar results.

Ability of primary auditory cortical neurons to detect amplitude modulation with rate and temporal codes: neurometric analysis.

Amplitude modulation (AM) is a common feature of natural sounds, and its detection is biologically important. Even though most sounds are not fully modulated, the majority of physiological studies have focused on fully modulated (100% modulation depth) sounds. We presented AM noise at a range of modulation depths to awake macaque monkeys while recording from neurons in primary auditory cortex (A1). The ability of neurons to detect partial AM with rate and temporal codes was assessed with signal detection methods. On average, single-cell synchrony was as or more sensitive than spike count in modulation detection. Cells are less sensitive to modulation depth if tested away from their best modulation frequency, particularly for temporal measures. Mean neural modulation detection thresholds in A1 are not as sensitive as behavioral thresholds, but with phase locking the most sensitive neurons are more sensitive, suggesting that for temporal measures the lower-envelope principle cannot account for thresholds. Three methods of preanalysis pooling of spike trains (multiunit, similar to convergence from a cortical column; within cell, similar to convergence of cells with matched response properties; across cell, similar to indiscriminate convergence of cells) all result in an increase in neural sensitivity to modulation depth for both temporal and rate codes. For the across-cell method, pooling of a few dozen cells can result in detection thresholds that approximate those of the behaving animal. With synchrony measures, indiscriminate pooling results in sensitive detection of modulation frequencies between 20 and 60 Hz, suggesting that differences in AM response phase are minor in A1.

Coding of amplitude modulation in primary auditory cortex.

Conflicting results have led to different views about how temporal modulation is encoded in primary auditory cortex (A1). Some studies find a substantial population of neurons that change firing rate without synchronizing to temporal modulation, whereas other studies fail to see these nonsynchronized neurons. As a result, the role and scope of synchronized temporal and nonsynchronized rate codes in AM processing in A1 remains unresolved. We recorded A1 neurons' responses in awake macaques to sinusoidal AM noise. We find most (37-78%) neurons synchronize to at least one modulation frequency (MF) without exhibiting nonsynchronized responses. However, we find both exclusively nonsynchronized neurons (7-29%) and "mixed-mode" neurons (13-40%) that synchronize to at least one MF and fire nonsynchronously to at least one other. We introduce new measures for modulation encoding and temporal synchrony that can improve the analysis of how neurons encode temporal modulation. These include comparing AM responses to the responses to unmodulated sounds, and a vector strength measure that is suitable for single-trial analysis. Our data support a transformation from a temporally based population code of AM to a rate-based code as information ascends the auditory pathway. The number of mixed-mode neurons found in A1 indicates this transformation is not yet complete, and A1 neurons may carry multiplexed temporal and rate codes.

Complex spectral interactions encoded by auditory cortical neurons: relationship between bandwidth and pattern.

The focus of most research on auditory cortical neurons has concerned the effects of rather simple stimuli, such as pure tones or broad-band noise, or the modulation of a single acoustic parameter. Extending these findings to feature coding in more complex stimuli such as natural sounds may be difficult, however. Generalizing results from the simple to more complex case may be complicated by non-linear interactions occurring between multiple, simultaneously varying acoustic parameters in complex sounds. To examine this issue in the frequency domain, we performed a parametric study of the effects of two global features, spectral pattern (here ripple frequency) and bandwidth, on primary auditory (A1) neurons in awake macaques. Most neurons were tuned for one or both variables and most also displayed an interaction between bandwidth and pattern implying that their effects were conditional or interdependent. A spectral linear filter model was able to qualitatively reproduce the basic effects and interactions, indicating that a simple neural mechanism may be able to account for these interdependencies. Our results suggest that the behavior of most A1 neurons is likely to depend on multiple parameters, and so most are unlikely to respond independently or invariantly to specific acoustic features.

Behavioral measures of auditory streaming in ferrets (Mustela putorius).

An important aspect of the analysis of auditory "scenes" relates to the perceptual organization of sound sequences into auditory "streams." In this study, we adapted two auditory perception tasks, used in recent human psychophysical studies, to obtain behavioral measures of auditory streaming in ferrets (Mustela putorius). One task involved the detection of shifts in the frequency of tones within an alternating tone sequence. The other task involved the detection of a stream of regularly repeating target tones embedded within a randomly varying multitone background. In both tasks, performance was measured as a function of various stimulus parameters, which previous psychophysical studies in humans have shown to influence auditory streaming. Ferret performance in the two tasks was found to vary as a function of these parameters in a way that is qualitatively consistent with the human data. These results suggest that auditory streaming occurs in ferrets, and that the two tasks described here may provide a valuable tool in future behavioral and neurophysiological studies of the phenomenon.

Adaptive, behaviorally gated, persistent encoding of task-relevant auditory information in ferret frontal cortex.

Top-down signals from frontal cortex are thought to be important in cognitive control of sensory processing. To explore this interaction, we compared activity in ferret frontal cortex and primary auditory cortex (A1) during auditory and visual tasks requiring discrimination between classes of reference and target stimuli. Frontal cortex responses were behaviorally gated, selectively encoded the timing and invariant behavioral meaning of target stimuli, could be rapid in onset, and sometimes persisted for hours following behavior. These results are consistent with earlier findings in A1 that attention triggered rapid, selective, persistent, task-related changes in spectrotemporal receptive fields. Simultaneously recorded local field potentials revealed behaviorally gated changes in inter-areal coherence that were selectively modulated between frontal cortex and focal regions of A1 that were responsive to target sounds. These results suggest that A1 and frontal cortex dynamically establish a functional connection during auditory behavior that shapes the flow of sensory information and maintains a persistent trace of recent task-relevant stimulus features.