FORUM: Remote testing for psychological and physiological acoustics.

Acoustics research involving human participants typically takes place in specialized laboratory settings. Listening studies, for example, may present controlled sounds using calibrated transducers in sound-attenuating or anechoic chambers. In contrast, remote testing takes place outside of the laboratory in everyday settings (e.g., participants' homes). Remote testing could provide greater access to participants, larger sample sizes, and opportunities to characterize performance in typical listening environments at the cost of reduced control of environmental conditions, less precise calibration, and inconsistency in attentional state and/or response behaviors from relatively smaller sample sizes and unintuitive experimental tasks. The Acoustical Society of America Technical Committee on Psychological and Physiological Acoustics launched the Task Force on Remote Testing (https://tcppasa.org/remotetesting/) in May 2020 with goals of surveying approaches and platforms available to support remote testing and identifying challenges and considerations for prospective investigators. The results of this task force survey were made available online in the form of a set of Wiki pages and summarized in this report. This report outlines the state-of-the-art of remote testing in auditory-related research as of August 2021, which is based on the Wiki and a literature search of papers published in this area since 2020, and provides three case studies to demonstrate feasibility during practice.

Distinct Representations of Tonotopy and Pitch in Human Auditory Cortex.

Frequency-to-place mapping, or tonotopy, is a fundamental organizing principle throughout the auditory system, from the earliest stages of auditory processing in the cochlea to subcortical and cortical regions. Although cortical maps are referred to as tonotopic, it is unclear whether they simply reflect a mapping of physical frequency inherited from the cochlea, a computation of pitch based on the fundamental frequency, or a mixture of these two features. We used high-resolution functional magnetic resonance imaging (fMRI) to measure BOLD responses as male and female human participants listened to pure tones that varied in frequency or complex tones that varied in either spectral content (brightness) or fundamental frequency (pitch). Our results reveal evidence for pitch tuning in bilateral regions that partially overlap with the traditional tonotopic maps of spectral content. In general, primary regions within Heschl's gyri (HGs) exhibited more tuning to spectral content, whereas areas surrounding HGs exhibited more tuning to pitch.SIGNIFICANCE STATEMENT Tonotopy, an orderly mapping of frequency, is observed throughout the auditory system. However, it is not known whether the tonotopy observed in the cortex simply reflects the frequency spectrum (as in the ear) or instead represents the higher-level feature of fundamental frequency, or pitch. Using carefully controlled stimuli and high-resolution functional magnetic resonance imaging (fMRI), we separated these features to study their cortical representations. Our results suggest that tonotopy in primary cortical regions is driven predominantly by frequency, but also reveal evidence for tuning to pitch in regions that partially overlap with the tonotopic gradients but extend into nonprimary cortical areas. In addition to resolving ambiguities surrounding cortical tonotopy, our findings provide evidence that selectivity for pitch is distributed bilaterally throughout auditory cortex.

The effects of data quantity on performance of temporal response function analyses of natural speech processing.

In recent years, temporal response function (TRF) analyses of neural activity recordings evoked by continuous naturalistic stimuli have become increasingly popular for characterizing response properties within the auditory hierarchy. However, despite this rise in TRF usage, relatively few educational resources for these tools exist. Here we use a dual-talker continuous speech paradigm to demonstrate how a key parameter of experimental design, the quantity of acquired data, influences TRF analyses fit to either individual data (subject-specific analyses), or group data (generic analyses). We show that although model prediction accuracy increases monotonically with data quantity, the amount of data required to achieve significant prediction accuracies can vary substantially based on whether the fitted model contains densely (e.g., acoustic envelope) or sparsely (e.g., lexical surprisal) spaced features, especially when the goal of the analyses is to capture the aspect of neural responses uniquely explained by specific features. Moreover, we demonstrate that generic models can exhibit high performance on small amounts of test data (2-8 min), if they are trained on a sufficiently large data set. As such, they may be particularly useful for clinical and multi-task study designs with limited recording time. Finally, we show that the regularization procedure used in fitting TRF models can interact with the quantity of data used to fit the models, with larger training quantities resulting in systematically larger TRF amplitudes. Together, demonstrations in this work should aid new users of TRF analyses, and in combination with other tools, such as piloting and power analyses, may serve as a detailed reference for choosing acquisition duration in future studies.

Effects of Age on Cortical Tracking of Word-Level Features of Continuous Competing Speech.

Speech-in-noise comprehension difficulties are common among the elderly population, yet traditional objective measures of speech perception are largely insensitive to this deficit, particularly in the absence of clinical hearing loss. In recent years, a growing body of research in young normal-hearing adults has demonstrated that high-level features related to speech semantics and lexical predictability elicit strong centro-parietal negativity in the EEG signal around 400 ms following the word onset. Here we investigate effects of age on cortical tracking of these word-level features within a two-talker speech mixture, and their relationship with self-reported difficulties with speech-in-noise understanding. While undergoing EEG recordings, younger and older adult participants listened to a continuous narrative story in the presence of a distractor story. We then utilized forward encoding models to estimate cortical tracking of four speech features: (1) word onsets, (2) "semantic" dissimilarity of each word relative to the preceding context, (3) lexical surprisal for each word, and (4) overall word audibility. Our results revealed robust tracking of all features for attended speech, with surprisal and word audibility showing significantly stronger contributions to neural activity than dissimilarity. Additionally, older adults exhibited significantly stronger tracking of word-level features than younger adults, especially over frontal electrode sites, potentially reflecting increased listening effort. Finally, neuro-behavioral analyses revealed trends of a negative relationship between subjective speech-in-noise perception difficulties and the model goodness-of-fit for attended speech, as well as a positive relationship between task performance and the goodness-of-fit, indicating behavioral relevance of these measures. Together, our results demonstrate the utility of modeling cortical responses to multi-talker speech using complex, word-level features and the potential for their use to study changes in speech processing due to aging and hearing loss.

Effects of noise precursors on the detection of amplitude and frequency modulation for tones in noise.

Recent studies on amplitude modulation (AM) detection for tones in noise reported that AM-detection thresholds improve when the AM stimulus is preceded by a noise precursor. The physiological mechanisms underlying this AM unmasking are unknown. One possibility is that adaptation to the level of the noise precursor facilitates AM encoding by causing a shift in neural rate-level functions to optimize level encoding around the precursor level. The aims of this study were to investigate whether such a dynamic-range adaptation is a plausible mechanism for the AM unmasking and whether frequency modulation (FM), thought to be encoded via AM, also exhibits the unmasking effect. Detection thresholds for AM and FM of tones in noise were measured with and without a fixed-level precursor. Listeners showing the unmasking effect were then tested with the precursor level roved over a wide range to modulate the effect of adaptation to the precursor level on the detection of the subsequent AM. It was found that FM detection benefits from a precursor and the magnitude of FM unmasking correlates with that of AM unmasking. Moreover, consistent with dynamic-range adaptation, the unmasking magnitude weakens as the level difference between the precursor and simultaneous masker of the tone increases.

Cortical Correlates of Attention to Auditory Features.

Pitch and timbre are two primary features of auditory perception that are generally considered independent. However, an increase in pitch (produced by a change in fundamental frequency) can be confused with an increase in brightness (an attribute of timbre related to spectral centroid) and vice versa. Previous work indicates that pitch and timbre are processed in overlapping regions of the auditory cortex, but are separable to some extent via multivoxel pattern analysis. Here, we tested whether attention to one or other feature increases the spatial separation of their cortical representations and if attention can enhance the cortical representation of these features in the absence of any physical change in the stimulus. Ten human subjects (four female, six male) listened to pairs of tone triplets varying in pitch, timbre, or both and judged which tone triplet had the higher pitch or brighter timbre. Variations in each feature engaged common auditory regions with no clear distinctions at a univariate level. Attending to one did not improve the separability of the neural representations of pitch and timbre at the univariate level. At the multivariate level, the classifier performed above chance in distinguishing between conditions in which pitch or timbre was discriminated. The results confirm that the computations underlying pitch and timbre perception are subserved by strongly overlapping cortical regions, but reveal that attention to one or other feature leads to distinguishable activation patterns even in the absence of physical differences in the stimuli.SIGNIFICANCE STATEMENT Although pitch and timbre are generally thought of as independent auditory features of a sound, pitch height and timbral brightness can be confused for one another. This study shows that pitch and timbre variations are represented in overlapping regions of auditory cortex, but that they produce distinguishable patterns of activation. Most importantly, the patterns of activation can be distinguished based on whether subjects attended to pitch or timbre even when the stimuli remained physically identical. The results therefore show that variations in pitch and timbre are represented by overlapping neural networks, but that attention to different features of the same sound can lead to distinguishable patterns of activation.

Orientation-selective contrast adaptation measured with SSVEP.

Exposure to oriented luminance contrast patterns causes a reduction in visual sensitivity specifically for the adapter orientation. This orientation selectivity is probably the most studied aspect of contrast adaptation, but it has rarely been measured with steady-state visually evoked potentials (SSVEPs), despite their becoming one of the more popular methods of human neuroscience. Here, we measured orientation selective adaptation by presenting a plaid stimulus of which the horizontal and vertical grating reversed contrast at different temporal frequencies, while recording EEG signals from occipital visual areas. In three experiments, we compared SSVEP responses to the plaid before and after adaptation. All experiments showed a significant decrease in SSVEP response at the frequency of the adapter orientation, whereas such an effect was absent for the frequency of the orthogonal orientation. Adaptation also led to robust phase delays, selectively for the SSVEP frequency corresponding to the adapter orientation. These results demonstrate the efficiency of SSVEPs for measuring orientation selective adaptation; the method can measure changes in both amplitude and phase, simultaneously for two orientations.

Beneficial Effects of Spatial Remapping for Reading With Simulated Central Field Loss.

Purpose: People with central field loss (CFL) lose information in the scotomatous region. Remapping is a method to modify images to present the missing information outside the scotoma. This study tested the hypothesis that remapping improves reading performance for subjects with simulated CFL. Methods: Circular central scotomas, with diameters ranging from 4° to 16°, were simulated in normally sighted subjects using an eye tracker on either a head-mounted display (HMD) (experiments 1, 2) or a traditional monitor (experiment 3). In the three experiments, reading speed was measured for groups of 7, 11, and 13 subjects with and without remapping of text. Results: Remapping increased reading speed in all three experiments. On the traditional monitor, it increased reading speed by 34% (8°), 38% (12°), and 35% (16°). In the two HMD experiments, remapping increased reading speed only for the largest scotoma size, possibly due to latency of updating of the simulated scotoma. Conclusions: Remapping significantly increased reading speed in simulated CFL subjects. Additional testing should examine the efficacy of remapping for reading and other visual tasks for patients with advanced CFL.

Equalization and decorrelation in primary visual cortex.

There are many theories on the purpose of neural adaptation, but evidence remains elusive. Here, we discuss the recent work by Benucci et al. (Nat Neurosci 16: 724-729, 2013), who measured for the first time the immediate effects of adaptation on the overall activity of a neuronal population. These measurements confirm two long-standing hypotheses about the purpose of adaptation, namely that adaptation counteracts biases in the statistics of the environment, and that it maintains decorrelation in neuronal stimulus selectivity.

Distinct mechanisms control contrast adaptation over different timescales.

Changes to the visual environment can happen at many timescales, from very transient to semi-permanent. To adapt optimally, the visual system also adjusts at different timescales, with longer-lasting environmental changes producing longer-lasting effects, but how the visual system adapts in this way remains unknown. Here, we show that contrast adaptation-the most-studied form of visual adaptation-has multiple controllers, each operating over a different time scale. In a series of experiments, subjects completed either a contrast matching, contrast detection, or tilt adjustment task, while adapting to contrast at one orientation. Following a relatively longer period (5 min) of adaptation to high contrast, subjects were "deadapted" for a shorter period (e.g., 40 s) to a lower contrast. Deadaptation eliminated perceptual aftereffects of adaptation, but continued testing in a neutral environment revealed their striking recovery. These results suggest the following account: Adaptation was controlled by at least two mechanisms, with initial adaptation affecting a longer-term one and deadaptation affecting a shorter-term one in the opposite direction. Immediately following deadaptation, the effects of the two mechanisms cancelled each other, but the short-term effects rapidly decayed, revealing ongoing longer-term adaptation. A single controlling mechanism cannot account for the observed recovery of effects, since once deadaptation cancels the initial longer-term adaptation, no trace of it remains. Combined with previous results at very long adaptation durations, the present results suggest that contrast adaptation is possibly controlled by a continuum of mechanisms acting over a large range of timescales.

Spontaneous recovery of motion and face aftereffects.

The ability of the visual system to rapidly adjust to changing environmental conditions is one of its key characteristics. Environmental changes can occur over a variety of timescales, however, and it remains unknown how the visual system adapts to these. Does a single mechanism control adaptation across all timescales, or is adaptation subserved by multiple mechanisms, each of which is tuned to its preferred duration? To address this question, we conducted three experiments in which subjects viewed motion (Exp. 1 and 2) or faces (Exp. 3) in a sequence designed to produce opposing aftereffects. A first adapter was presented for a relatively long duration, while a second one was presented only long enough to extinguish the effects of the initial adapter. Continued measurement of aftereffects revealed a spontaneous recovery of adaptation caused by the initial, longer-lasting adapter in all three experiments. This pattern of results suggests that adaptation in the visual system generally reflects a combination of multiple temporally-tuned mechanisms.

High temporal resolution decoding of object position and category.

We effortlessly and seemingly instantaneously recognize thousands of objects, although we rarely--if ever--see the same image of an object twice. The retinal image of an object can vary by context, size, viewpoint, illumination, and location. The present study examined how the visual system abstracts object category across variations in retinal location. In three experiments, participants viewed images of objects presented to different retinal locations while brain activity was recorded using magnetoencephalography (MEG). A pattern classifier was trained to recover the stimulus position (Experiments 1, 2, and 3) and category (Experiment 3) from the recordings. Using this decoding approach, we show that an object's location in the visual field can be recovered in high temporal resolution (5 ms) and with sufficient fidelity to capture topographic organization in visual areas. Experiment 3 showed that an object's category could be recovered from the recordings as early as 135 ms after the onset of the stimulus and that category decoding generalized across retinal location (i.e., position invariance). Our experiments thus show that the visual system rapidly constructs a category representation for objects that is invariant to position.