Training to Improve Spatial Hearing and Situation Awareness when Wearing Hearing Protection.

INTRODUCTION: Hearing protection devices (HPDs) are standard personal protective equipment in military settings, but many service members may choose to not use HPDs because they impair spatial hearing and situation awareness. In an effort to reduce barriers to compliance by improving situation awareness while wearing HPDs, this study investigated whether brief training could counteract spatial hearing deficits when wearing HPDs. Participant's ability to correctly apply the HPDs across days was also examined. MATERIALS AND METHODS: Young adults were randomly assigned to one of two groups: training or control (n = 25/group). Participants in each group performed a spatial hearing task while wearing HPDs and in an open ear condition without HPDs. Individual targets were battlefield sounds or white noise presented from a speaker array that surrounded the participant in the horizontal plane. After presentation of each target sound, the participant then controlled a white noise "auditory pointer," which they moved to the perceived location of the target. The two primary measures were the percent of trials with very large errors (> 45°), which were usually due to confusing front and back locations, and absolute localization, which is the difference between the pointer location and the true sound location. Both groups were tested on Days 1 (baseline) and 5 (post-test). On Days 2 to 4, the training group wore HPDs while receiving auditory and visual feedback after each trial. RESULTS: Across all participants on Day 1, wearing HPDs increased the frequency of very large errors by about 3× and impaired localization by about 40%, relative to the open ear condition. When comparing performance at baseline (Day 1) and post-training Day 5, the training group with HPDs had significant reductions in very large errors and improved absolute localization (P values < .001). The training group also had significant improvements from Days 1 to 5 in the open ear condition. When the control group wore HPDs, there were also significant improvements from Days 1 to 5 (fewer very large errors and better localization), with smaller effect sizes vs. the training group. Controls did not have significant improvement in the open ear condition, but had similar trends. Most participants consistently applied the HPDs, but a subset of ∼20% frequently failed to achieve the criterion attenuation of 15 dB (over 0.25-4.0 kHz) in both ears. CONCLUSIONS: These findings show that simple, relatively brief practice and training can substantially reduce HPD impairments on spatial hearing and situation awareness. The gains from training and practice can inform the development of relatively simple, brief methods to reduce HPD spatial hearing impairments, potentially leading to increased HPD compliance. Longitudinal data show that a subset of participants would not have received the full benefit of hearing protection because of improper application of the HPDs.

A novel non-word speech preparation task to increase stuttering frequency in experimental settings for longitudinal research.

PURPOSE: The variable and intermittent nature of stuttering makes it difficult to consistently elicit a sufficient number of stuttered trials for longitudinal experimental research. This study tests the efficacy of using non-word pairs that phonetically mimic English words with no associated meaning, to reliably elicit balanced numbers of stuttering and fluent trials over multiple sessions. The study also evaluated the effect of non-word length on stuttering frequency, the consistency of stuttering frequency across sessions, and potential carry-over effects of increased stuttering frequency in the experimental task to conversational and reading speech after the task. METHODS: Twelve adults who stutter completed multiple sessions (mean of 4.8 sessions) where they were video-recorded during pre-task reading and conversation, followed by an experimental task where they read 400 non-word pairs randomized for each session, and then a post-task reading and conversation sample. RESULTS: On average, across sessions and participants, non-word pairs consistently yielded a balanced distribution of fluent (60.7%) and stuttered (39.3%) trials over five sessions. Non-word length had a positive effect on stuttering frequency. No carryover effects from experimental to post-task conversation and reading were found. CONCLUSIONS: Non-word pairs effectively and consistently elicited balanced proportions of stuttered and fluent trials. This approach can be used to gather longitudinal data to better understand the neurophysiological and behavioral correlates of stuttering.

Multimodal explainable AI predicts upcoming speech behavior in adults who stutter.

A key goal of cognitive neuroscience is to better understand how dynamic brain activity relates to behavior. Such dynamics, in terms of spatial and temporal patterns of brain activity, are directly measured with neurophysiological methods such as EEG, but can also be indirectly expressed by the body. Autonomic nervous system activity is the best-known example, but, muscles in the eyes and face can also index brain activity. Mostly parallel lines of artificial intelligence research show that EEG and facial muscles both encode information about emotion, pain, attention, and social interactions, among other topics. In this study, we examined adults who stutter (AWS) to understand the relations between dynamic brain and facial muscle activity and predictions about future behavior (fluent or stuttered speech). AWS can provide insight into brain-behavior dynamics because they naturally fluctuate between episodes of fluent and stuttered speech behavior. We focused on the period when speech preparation occurs, and used EEG and facial muscle activity measured from video to predict whether the upcoming speech would be fluent or stuttered. An explainable self-supervised multimodal architecture learned the temporal dynamics of both EEG and facial muscle movements during speech preparation in AWS, and predicted fluent or stuttered speech at 80.8% accuracy (chance=50%). Specific EEG and facial muscle signals distinguished fluent and stuttered trials, and systematically varied from early to late speech preparation time periods. The self-supervised architecture successfully identified multimodal activity that predicted upcoming behavior on a trial-by-trial basis. This approach could be applied to understanding the neural mechanisms driving variable behavior and symptoms in a wide range of neurological and psychiatric disorders. The combination of direct measures of neural activity and simple video data may be applied to developing technologies that estimate brain state from subtle bodily signals.

Auditory spatial attention gradients and cognitive control as a function of vigilance.

Selection and effort are central to attention, yet it is unclear whether they draw on a common pool of cognitive resources, and if so, whether there are differences for early versus later stages of cognitive processing. This study assessed effort by quantifying the vigilance decrement, and spatial processing at early and later stages as a function of time-on-task. Participants performed an auditory spatial attention task, with occasional "catch" trials requiring no response. Psychophysiological measures included bilateral cerebral blood flow (transcranial Doppler), pupil dilation, and blink rate. The shape of attention gradients using reaction time indexed early processing, and did not significantly vary over time. Later stimulus-response conflict was comparable over time, except for a reduction to left hemispace stimuli. Target and catch trial accuracy decreased with time, with a more abrupt decrease for catch versus target trials. Diffusion decision modeling found progressive decreases in information accumulation rate and non-decision time, and the adoption of more liberal response criteria. Cerebral blood flow increased from baseline and then decreased over time, particularly in the left hemisphere. Blink rate steadily increased over time, while pupil dilation increased only at the beginning and then returned towards baseline. The findings suggest dissociations between resources for selectivity and effort. Measures of high subjective effort and temporal declines in catch trial accuracy and cerebral blood flow velocity suggest a standard vigilance decrement was evident in parallel with preserved selection. Different attentional systems and classes of computations that may account for dissociations between selectivity versus effort are discussed.

Interpretable Self-Supervised Facial Micro-Expression Learning to Predict Cognitive State and Neurological Disorders.

Human behavior is the confluence of output from voluntary and involuntary motor systems. The neural activities that mediate behavior, from individual cells to distributed networks, are in a state of constant flux. Artificial intelligence (AI) research over the past decade shows that behavior, in the form of facial muscle activity, can reveal information about fleeting voluntary and involuntary motor system activity related to emotion, pain, and deception. However, the AI algorithms often lack an explanation for their decisions, and learning meaningful representations requires large datasets labeled by a subject-matter expert. Motivated by the success of using facial muscle movements to classify brain states and the importance of learning from small amounts of data, we propose an explainable self-supervised representation-learning paradigm that learns meaningful temporal facial muscle movement patterns from limited samples. We validate our methodology by carrying out comprehensive empirical study to predict future speech behavior in a real-world dataset of adults who stutter (AWS). Our explainability study found facial muscle movements around the eyes (p <0.×001) and lips (p <0.001) differ significantly before producing fluent vs. disfluent speech. Evaluations using the AWS dataset demonstrates that the proposed self-supervised approach achieves a minimum of 2.51% accuracy improvement over fully-supervised approaches.

Numbers in short-term memory bias auditory spatial perception.

The cognitive penetration literature suggests that top-down knowledge influences perception, but whether such influences exist is controversial. We tested for top-down influences on perception by loading short-term memory with digits and then had participants make perceptual judgments to index spatial hearing. Memory of spatial number codes were predicted to bias spatial judgments to the left for small digits and rightward for larger digits. Participants encoded one or more digits and then made spatial judgments in either spatial hearing or dichotic listening tasks. Results across five experiments supported the predicted spatial biases. Digits had to be deliberately encoded, and at least two were needed to be memorized before a small number left-right bias in dichotic listening was evident. In dichotic listening, smaller numbers in memory also promoted more intrusions, and a mix of small and large numbers enhanced the right ear advantage. Results suggest that long-term knowledge about number magnitude imparts a top-down bias on auditory spatial perception. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Sensorimotor Integration Can Enhance Auditory Perception.

Whenever we move, speak, or play musical instruments, our actions generate auditory sensory input. The sensory consequences of our actions are thought to be predicted via sensorimotor integration, which involves anatomical and functional links between auditory and motor brain regions. The physiological connections are relatively well established, but less is known about how sensorimotor integration affects auditory perception. The sensory attenuation hypothesis suggests that the perceived loudness of self-generated sounds is attenuated to help distinguish self-generated sounds from ambient sounds. Sensory attenuation would work for louder ambient sounds, but could lead to less accurate perception if the ambient sounds were quieter. We hypothesize that a key function of sensorimotor integration is the facilitated processing of self-generated sounds, leading to more accurate perception under most conditions. The sensory attenuation hypothesis predicts better performance for higher but not lower intensity comparisons, whereas sensory facilitation predicts improved perception regardless of comparison sound intensity. A series of experiments tested these hypotheses, with results supporting the enhancement hypothesis. Overall, people were more accurate at comparing the loudness of two sounds when making one of the sounds themselves. We propose that the brain selectively modulates the perception of self-generated sounds to enhance representations of action consequences.

Dynamics of auditory spatial attention gradients.

Auditory spatial attention faces the conflicting demands of focusing on the current task while also rapidly shifting location to unexpected stimuli. We examined the interplay of sustained focus and intermittent shifts in an auditory spatial attention task. Most trials required a choice response from a standard location in virtual space (L-R: -90°, 0°, +90°), but occasionally the location shifted between 45°-180°. Reaction time curves for angular shifts had a quadratic shape, with slowing for small shifts but faster reaction times for larger shifts. The reaction time curves were maintained at faster stimulus rates and usually scaled to fit the range of stimulus locations. However, focus on the right had an attenuated curve, and did not scale to the range of locations. The findings suggest two mechanisms: a top-down bias centered on standard locations that decreases with distance, and a bottom-up bias that under these conditions increases with distance from the standard location.

Auditory spatial attention capture, disengagement, and response selection in normal aging.

Attention control is a core element of cognitive aging, but the specific mechanisms that differ with age are unclear. Here we used a novel auditory spatial attention task to evaluate stimulus processing at the level of early attention capture, later response selection, and the lingering effects of attention capture across trials in young and older adults. We found that the shapes of spatial attention capture gradients were remarkably similar in young and older adults, but only the older group had lingering effects of attention capture on the next trial. Response selection for stimulus-response incompatibilities took longer in older subjects, but primarily when attending to the midline location. The results suggest that the likelihood and spatial tuning of attention capture is comparable among groups, but once attention is captured, older subjects take longer to disengage. Age differences in response selection were supported, but may not be a general feature of cognitive aging.

Single-Trial Classification of Disfluent Brain States in Adults Who Stutter.

Normal human speech requires precise coordination between motor planning and sensory processing. Speech disfluencies are common when children learn to talk, but usually abate with time. About 5% of children experience stuttering. For most, this resolves within a year. However, for approximately 1% of the world population, stuttering continues into adulthood, which is termed 'persistent developmental stuttering'. Most stuttering events occur at the beginning of an utterance. So, in principle, brain activity before speaking should differ between fluent and stuttered speech. Here we present a method for classifying brain network states associated with fluent vs. stuttered speech on a single trial basis. Brain activity was recorded with EEG before people who stutter read aloud pseudo-word pairs. Offline independent component analysis (ICA) was used to identify the independent neural sources that underlie speech preparation. A time window selection algorithm extracted spectral power and coherence data from salient windows specific to each neural source. A stepwise linear discriminant analysis (sLDA) algorithm predicted fluent vs. stuttered speech for 81% of trials in two subjects. These results support the feasibility of developing a brain-computer interface (BCI) system to detect stuttering before it occurs, with potential for therapeutic application.

Unilateral repetitive transcranial magnetic stimulation differentially affects younger and older adults completing a verbal working memory task.

Functional neuroimaging studies have found that lateralization of activity in the dorsolateral prefrontal cortex (dlPFC) is reduced with aging. In the present study, repetitive transcranial magnetic stimulation (rTMS) was used to disrupt dlPFC activity in order to test the relationship of dlPFC laterality and age in verbal working memory (WM). Young (n=36) and older (n=35) subjects received 1Hz-rTMS (sham or active) to left or right dlPFC and WM performance was assessed pre- and post-stimulation via the n-back task. Significant increases in WM accuracy were observed following rTMS in the right dlPFC and sham conditions, but not with the left dlPFC stimulation. This was accompanied by a decrease in left P1 latency was also observed following left dlPFC stimulation. In contrast, older adults did not show a disruption in WM performance following rTMS in any of the stimulation conditions and exhibited increased left P3 amplitude following left stimulation. Our results show that changes in prefrontal laterality are evident with increasing age (left stimulation affects younger adults while older adults are not affected by stimulation) and this change is associated with specific neurophysiologic measures.

Numerical value biases sound localization.

Speech recognition starts with representations of basic acoustic perceptual features and ends by categorizing the sound based on long-term memory for word meaning. However, little is known about whether the reverse pattern of lexical influences on basic perception can occur. We tested for a lexical influence on auditory spatial perception by having subjects make spatial judgments of number stimuli. Four experiments used pointing or left/right 2-alternative forced choice tasks to examine perceptual judgments of sound location as a function of digit magnitude (1-9). The main finding was that for stimuli presented near the median plane there was a linear left-to-right bias for localizing smaller-to-larger numbers. At lateral locations there was a central-eccentric location bias in the pointing task, and either a bias restricted to the smaller numbers (left side) or no significant number bias (right side). Prior number location also biased subsequent number judgments towards the opposite side. Findings support a lexical influence on auditory spatial perception, with a linear mapping near midline and more complex relations at lateral locations. Results may reflect coding of dedicated spatial channels, with two representing lateral positions in each hemispace, and the midline area represented by either their overlap or a separate third channel.

Impact of Spatial and Verbal Short-Term Memory Load on Auditory Spatial Attention Gradients.

Short-term memory load can impair attentional control, but prior work shows that the extent of the effect ranges from being very general to very specific. One factor for the mixed results may be reliance on point estimates of memory load effects on attention. Here we used auditory attention gradients as an analog measure to map-out the impact of short-term memory load over space. Verbal or spatial information was maintained during an auditory spatial attention task and compared to no-load. Stimuli were presented from five virtual locations in the frontal azimuth plane, and subjects focused on the midline. Reaction times progressively increased for lateral stimuli, indicating an attention gradient. Spatial load further slowed responses at lateral locations, particularly in the left hemispace, but had little effect at midline. Verbal memory load had no (Experiment 1), or a minimal (Experiment 2) influence on reaction times. Spatial and verbal load increased switch costs between memory encoding and attention tasks relative to the no load condition. The findings show that short-term memory influences the distribution of auditory attention over space; and that the specific pattern depends on the type of information in short-term memory.

Cortical activity during cued picture naming predicts individual differences in stuttering frequency.

OBJECTIVE: Developmental stuttering is characterized by fluent speech punctuated by stuttering events, the frequency of which varies among individuals and contexts. Most stuttering events occur at the beginning of an utterance, suggesting neural dynamics associated with stuttering may be evident during speech preparation. METHODS: This study used EEG to measure cortical activity during speech preparation in men who stutter, and compared the EEG measures to individual differences in stuttering rate as well as to a fluent control group. Each trial contained a cue followed by an acoustic probe at one of two onset times (early or late), and then a picture. There were two conditions: a speech condition where cues induced speech preparation of the picture's name and a control condition that minimized speech preparation. RESULTS: Across conditions stuttering frequency correlated to cue-related EEG beta power and auditory ERP slow waves from early onset acoustic probes. CONCLUSIONS: The findings reveal two new cortical markers of stuttering frequency that were present in both conditions, manifest at different times, are elicited by different stimuli (visual cue, auditory probe), and have different EEG responses (beta power, ERP slow wave). SIGNIFICANCE: The cue-target paradigm evoked brain responses that correlated to pre-experimental stuttering rate.

The Role of Right Inferior Parietal Cortex in Auditory Spatial Attention: A Repetitive Transcranial Magnetic Stimulation Study.

Behavioral studies support the concept of an auditory spatial attention gradient by demonstrating that attentional benefits progressively diminish as distance increases from an attended location. Damage to the right inferior parietal cortex can induce a rightward attention bias, which implicates this region in the construction of attention gradients. This study used event-related potentials (ERPs) to define attention-related gradients before and after repetitive transcranial magnetic stimulation (rTMS) to the right inferior parietal cortex. Subjects (n = 16) listened to noise bursts at five azimuth locations (left to right: -90°, -45°, 0° midline, +45°, +90°) and responded to stimuli at one target location (-90°, +90°, separate blocks). ERPs as a function of non-target location were examined before (baseline) and after 0.9 Hz rTMS. Results showed that ERP attention gradients were observed in three time windows (frontal 230-340, parietal 400-460, frontal 550-750 ms). Significant transient rTMS effects were seen in the first and third windows. The first window had a voltage decrease at the farthest location when attending to either the left or right side. The third window had on overall increase in positivity, but only when attending to the left side. These findings suggest that rTMS induced a small contraction in spatial attention gradients within the first time window. The asymmetric effect of attended location on gradients in the third time window may relate to neglect of the left hemispace after right parietal injury. Together, these results highlight the role of the right inferior parietal cortex in modulating frontal lobe attention network activity.

Speech preparation in adults with persistent developmental stuttering.

Motor efference copy conveys movement information to sensory areas before and during vocalization. We hypothesized speech preparation would modulate auditory processing, via motor efference copy, differently in men who stutter (MWS) vs. fluent adults. Participants (n=12/group) had EEG recorded during a cue-target paradigm with two conditions: speech which allowed for speech preparation, while a control condition did not. Acoustic stimuli probed auditory responsiveness between the cue and target. MWS had longer vocal reaction times (p<0.01) when the cue-target differed (10% of trials), suggesting a difficulty of rapidly updating their speech plans. Acoustic probes elicited a negative slow wave indexing motor efference copy that was smaller in MWS vs. fluent adults (p<0.03). Current density responses in MWS showed smaller left prefrontal responses and auditory responses that were delayed and correlated to stuttering rate. Taken together, the results provide insight into the cortical mechanisms underlying atypical speech planning and dysfluencies in MWS.

Rapid cortical dynamics associated with auditory spatial attention gradients.

Behavioral and EEG studies suggest spatial attention is allocated as a gradient in which processing benefits decrease away from an attended location. Yet the spatiotemporal dynamics of cortical processes that contribute to attentional gradients are unclear. We measured EEG while participants (n = 35) performed an auditory spatial attention task that required a button press to sounds at one target location on either the left or right. Distractor sounds were randomly presented at four non-target locations evenly spaced up to 180° from the target location. Attentional gradients were quantified by regressing ERP amplitudes elicited by distractors against their spatial location relative to the target. Independent component analysis was applied to each subject's scalp channel data, allowing isolation of distinct cortical sources. Results from scalp ERPs showed a tri-phasic response with gradient slope peaks at ~300 ms (frontal, positive), ~430 ms (posterior, negative), and a plateau starting at ~550 ms (frontal, positive). Corresponding to the first slope peak, a positive gradient was found within a central component when attending to both target locations and for two lateral frontal components when contralateral to the target location. Similarly, a central posterior component had a negative gradient that corresponded to the second slope peak regardless of target location. A right posterior component had both an ipsilateral followed by a contralateral gradient. Lateral posterior clusters also had decreases in α and ß oscillatory power with a negative slope and contralateral tuning. Only the left posterior component (120-200 ms) corresponded to absolute sound location. The findings indicate a rapid, temporally-organized sequence of gradients thought to reflect interplay between frontal and parietal regions. We conclude these gradients support a target-based saliency map exhibiting aspects of both right-hemisphere dominance and opponent process models.

Atypical caudate anatomy in children who stutter.

A temporal motor defect in speech preparation and/or planning may contribute to the development of stuttering. This defect may be linked to a dysfunctional cortical-subcortical network at the level of the striatum. To determine whether structural differences exist and whether group differences are associated with stuttering severity or manual laterality, the caudate was measured in 14 children who stutter (CWS) and in a control group of right-handed boys, ages 8-13 years. There was a statistically significant hemisphere by group effect for caudate volume. CWS had reduced right caudate volume and atypical leftward asymmetry compared to controls. Nine of the 13 CWS with atypical caudate asymmetry had atypical manual laterality. These anomalies may represent a vulnerability that perturbs speech planning/preparation and contributes to inefficiencies in action-perception coupling that may be an indicator of stuttering susceptibility. These results suggest that right-handed boys who stutter may have a defect in the feedforward cortico-striato-thalamo-cortical networks.

The SpeechEasy device in stuttering and nonstuttering adults: fluency effects while speaking and reading.

The SpeechEasy is an electronic device designed to alleviate stuttering by manipulating auditory feedback via time delays and frequency shifts. Device settings (control, default, custom), ear-placement (left, right), speaking task, and cognitive variables were examined in people who stutter (PWS) (n=14) compared to controls (n=10). Among the PWS there was a significantly greater reduction in stuttering (compared to baseline) with custom device settings compared to the non-altered feedback (control) condition. Stuttering was reduced the most during reading, followed by narrative and conversation. For the conversation task, stuttering was reduced more when the device was worn in the left ear. Those individuals with a more severe stuttering rate at baseline had a greater benefit from the use of the device compared to individuals with less severe stuttering. Our results support the view that overt stuttering is associated with defective speech-language monitoring that can be influenced by manipulating auditory feedback.