Combining Cardiovascular and Pupil Features Using k-Nearest Neighbor Classifiers to Assess Task Demand, Social Context, and Sentence Accuracy During Listening.

In daily life, both acoustic factors and social context can affect listening effort investment. In laboratory settings, information about listening effort has been deduced from pupil and cardiovascular responses independently. The extent to which these measures can jointly predict listening-related factors is unknown. Here we combined pupil and cardiovascular features to predict acoustic and contextual aspects of speech perception. Data were collected from 29 adults (mean  =  64.6 years, SD  =  9.2) with hearing loss. Participants performed a speech perception task at two individualized signal-to-noise ratios (corresponding to 50% and 80% of sentences correct) and in two social contexts (the presence and absence of two observers). Seven features were extracted per trial: baseline pupil size, peak pupil dilation, mean pupil dilation, interbeat interval, blood volume pulse amplitude, pre-ejection period and pulse arrival time. These features were used to train k-nearest neighbor classifiers to predict task demand, social context and sentence accuracy. The k-fold cross validation on the group-level data revealed above-chance classification accuracies: task demand, 64.4%; social context, 78.3%; and sentence accuracy, 55.1%. However, classification accuracies diminished when the classifiers were trained and tested on data from different participants. Individually trained classifiers (one per participant) performed better than group-level classifiers: 71.7% (SD  =  10.2) for task demand, 88.0% (SD  =  7.5) for social context, and 60.0% (SD  =  13.1) for sentence accuracy. We demonstrated that classifiers trained on group-level physiological data to predict aspects of speech perception generalized poorly to novel participants. Individually calibrated classifiers hold more promise for future applications.

Copresence Was Found to Be Related to Some Pupil Measures in Persons With Hearing Loss While They Performed a Speech-in-Noise Task.

OBJECTIVES: To assess if a manipulation of copresence was related to speech-in-noise task performance, arousal, and effort of persons with hearing loss. Task-related arousal and effort were measured by means of pupillometry. DESIGN: Twenty-nine participants (mean age: 64.6 years) with hearing loss (4-frequency pure-tone average [4F-PTA] of 50.2 dB HL [SD = 8.9 dB] in the right ear and 51.3 dB HL [SD = 8.7 dB] in the left ear; averaged across 0.5, 1, 2, and 4 kHz) listened to and repeated spoken Danish sentences that were masked by four streams of continuous speech. Participants were presented with blocks of 20 sentences, during which copresence was manipulated by having participants do the task either alone or accompanied by two observers who were recruited from a similar age group. The task was presented at two difficulty levels, which was accomplished by fixing the signal-to-noise ratio of the speech and masker to match the thresholds at which participants were estimated to correctly repeat 50% (difficult) or 80% (easy) of the sentences in a block. Performance was assessed based on whether or not sentences were repeated correctly. Measures of pupil size (baseline pupil size [BPS], peak pupil dilation [PPD], and mean pupil dilation [MPD]) were used to index arousal and effort. Participants also completed ratings of subjective effort and stress after each block of sentences and a self-efficacy for listening-questionnaire. RESULTS: Task performance was not associated with copresence, but was found to be related to 4F-PTA. An increase in BPS was found for copresence conditions, compared to alone conditions. Furthermore, a post-hoc exploratory analysis revealed that the copresence conditions were associated with a significantly larger pupil size in the second half of the task-evoked pupil response (TEPR). No change in PPD or MPD did was detected between copresence and alone conditions. Self-efficacy, 4F-PTA, and age were not found to be related to the pupil data. Subjective ratings were sensitive to task difficulty but not copresence. CONCLUSION: Copresence was not found to be related to speech-in-noise performance, PPD, or MPD in persons with HL but was associated with an increase in arousal (as indicated by a larger BPS). This could be related to premobilization of effort and/or discomfort in response to the observers' presence. Furthermore, an exploratory analysis of the pupil data showed that copresence was associated with greater pupil dilations in the second half of the TEPR. This may indicate that participants invested more effort during the speech-in-noise task while in the presence of the observers, but that this increase in effort may not necessarily have been related to listening itself. Instead, other speech-in-noise task-related processes, such as preparing to respond, could have been influenced by copresence.

Social observation increases the cardiovascular response of hearing-impaired listeners during a speech reception task.

Certain cardiovascular measures allow for distinction between sympathetic and parasympathetic nervous system activity. Applied during listening, these measures may provide a novel and complementary insight into listening effort. To date, few studies have implemented cardiovascular measures of listening effort and seldom have these included hearing-impaired participants. These studies have generally measured changes in cardiovascular parameters while manipulating environmental factors, such as listening difficulty. Yet, listening effort is also known to be moderated by individual factors, including the importance of performing successfully. In this study, we aimed to manipulate success importance by adding observers to the traditional laboratory set-up. Twenty-nine hearing-impaired participants performed a speech reception task both alone and in the presence of two observers. Auditory stimuli consisted of Danish Hearing in Noise Test (HINT) sentences masked by four-talker babble. Sentences were delivered at two individually adapted signal-to-noise ratios, corresponding to 50 and 80% of sentences correct. We measured change scores, relative to baseline, of pre-ejection period, two indices of heart rate variability, heart rate and blood pressure (systolic, diastolic, and mean arterial pressure). After each condition, participants rated their effort investment, stress, tendency to give up and preference to change the situation to improve audibility. A multivariate analysis revealed that cardiovascular reactivity increased in the presence of the observers, compared to when the task was performed alone. More specifically, systolic, diastolic, and mean arterial blood pressure increased while observed. Interestingly, participants' subjective ratings were sensitive only to intelligibility level, not the observation state. This study was the first to report results from a range of different cardiovascular variables measured from hearing-impaired participants during a speech reception task. Due to the timing of the observers' presence, we were not able to conclusively attribute these physiological changes to being task related. Therefore, instead of representing listening effort, we suggest that the increased cardiovascular response detected during observation reveals increased physiological stress associated with potential evaluation.

Investigating the Influences of Task Demand and Reward on Cardiac Pre-Ejection Period Reactivity During a Speech-in-Noise Task.

OBJECTIVES: Effort investment during listening varies as a function of task demand and motivation. Several studies have manipulated both these factors to elicit and measure changes in effort associated with listening. The cardiac pre-ejection period (PEP) is a relatively novel measure in the field of cognitive hearing science. This measure, which reflects sympathetic nervous system activity on the heart, has previously been implemented during a tone discrimination task but not during a speech-in-noise task. Therefore, the primary goal of this study was to explore the influences of signal to noise ratio (SNR) and monetary reward level on PEP reactivity during a speech-in-noise task. DESIGN: Thirty-two participants with normal hearing (mean age = 22.22 years, SD = 3.03) were recruited at VU University Medical Center. Participants completed a Dutch speech-in-noise test with a single-interfering-talker masking noise. Six fixed SNRs, selected to span the entire psychometric performance curve, were presented in a block-wise fashion. Participants could earn a low (€0.20) or high (€5.00) reward by obtaining a score of ≥70% of words correct in each block. The authors analyzed PEP reactivity: the change in PEP measured during the task, relative to the baseline during rest. Two separate methods of PEP analysis were used, one including data from the whole task block and the other including data obtained during presentation of the target sentences only. After each block, participants rated their effort investment, performance, tendency to give up, and the perceived difficulty of the task. They also completed the need for recovery questionnaire and the reading span test, which are indices of additional factors (fatigue and working memory capacity, respectively) that are known to influence listening effort. RESULTS: Average sentence perception scores ranged from 2.73 to 91.62%, revealing a significant effect of SNR. In addition, an improvement in performance was elicited by the high, compared to the low reward level. A linear relationship between SNR and PEP reactivity was demonstrated: at the lower SNRs PEP reactivity was the most negative, indicating greater effort investment compared to the higher SNRs. The target stimuli method of PEP analysis was more sensitive to this effect than the block-wise method. Contrary to expectations, no significant impact of reward on PEP reactivity was found in the present dataset. Also, there was no physiological evidence that participants were disengaged, even when performance was poor. A significant correlation between need for recovery scores and average PEP reactivity was demonstrated, indicating that a lower need for recovery was associated with less effort investment. CONCLUSIONS: This study successfully implemented the measurement of PEP during a standard speech-in-noise test and included two distinct methods of PEP analysis. The results revealed for the first time that PEP reactivity varies linearly with task demand during a speech-in-noise task, although the effect size was small. No effect of reward on PEP was demonstrated. Finally, participants with a higher need for recovery score invested more effort, as shown by average PEP reactivity, than those with a lower need for recovery score.