Changing-state irrelevant speech disrupts visual-verbal but not visual-spatial serial recall.

In an influential article, Jones et al. (1995) provide evidence that auditory distraction by changing relative to repetitive auditory distracters (the changing-state effect) did not differ between a visual-verbal and visual-spatial serial recall task, providing evidence for an amodal mechanism for the representation of serial order in short-term memory that transcends modalities. This finding has been highly influential for theories of short-term memory and auditory distraction. However, evidence vis-à-vis the robustness of this result is sorely lacking. Here, two high-powered replications of Jones et al.'s (1995) crucial Experiment 4 were undertaken. In the first partial replication (n = 64), a fully within-participants design was adopted, wherein participants undertook both the visual-verbal and visual-spatial serial recall tasks under different irrelevant sound conditions, without a retention period. The second near-identical replication (n = 128), incorporated a retention period and implemented the task-modality manipulation as a between-participants factor, as per the original Jones et al. (1995; Experiment 4) study. In both experiments, the changing-state effect was observed for visual-verbal serial recall but not for visual-spatial serial recall. The results are consistent with modular and interference-based accounts of distraction and challenge some aspects of functional equivalence accounts. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

A lecturer's voice quality and its effect on memory, listening effort, and perception in a VR environment.

Many lecturers develop voice problems, such as hoarseness. Nevertheless, research on how voice quality influences listeners' perception, comprehension, and retention of spoken language is limited to a small number of audio-only experiments. We aimed to address this gap by using audio-visual virtual reality (VR) to investigate the impact of a lecturer's hoarseness on university students' heard text recall, listening effort, and listening impression. Fifty participants were immersed in a virtual seminar room, where they engaged in a Dual-Task Paradigm. They listened to narratives presented by a virtual female professor, who spoke in either a typical or hoarse voice. Simultaneously, participants performed a secondary task. Results revealed significantly prolonged secondary-task response times with the hoarse voice compared to the typical voice, indicating increased listening effort. Subjectively, participants rated the hoarse voice as more annoying, effortful to listen to, and impeding for their cognitive performance. No effect of voice quality was found on heard text recall, suggesting that, while hoarseness may compromise certain aspects of spoken language processing, this might not necessarily result in reduced information retention. In summary, our findings underscore the importance of promoting vocal health among lecturers, which may contribute to enhanced listening conditions in learning spaces.

The effects of irrelevant speech on physiological stress, cognitive performance, and subjective experience - Focus on heart rate variability.

Irrelevant speech impairs cognitive performance, especially in tasks requiring verbal short-term memory. Working on these tasks during irrelevant speech can also cause a physiological stress reaction. The aim of this study was to examine heart rate variability (HRV) as a non-invasive and easy-to-use stress measure in an irrelevant speech paradigm. Thirty participants performed cognitive tasks (n-back and serial recall) during two sound conditions: irrelevant speech (50 dB) and quiet (33 dB steady-state noise). The influence of conditions as well as presentation orders of conditions were examined on performance, subjective experience, and physiological stress. Working during irrelevant speech compared to working during quiet reduced performance, namely accuracy, in the serial recall task. It was more annoying, heightened the perceived workload, and lowered acoustic satisfaction. It was related to higher physiological stress by causing faster heart rate and changes in HRV frequency-domain analysis (LF, HF and LF/HF). The order of conditions showed some additional effects. When speech was the first condition, 3-back performance was less accurate, and serial recall response times were longer, heart rate was faster, and successive heart beats had less variability (lower RMSSD) during speech than during quiet. When quiet was the first condition, heart rate was faster and reaction times in 3-back were slower during quiet than during speech. The negative effect of irrelevant speech was clear in experience, performance, and physiological stress. The study shows that HRV can be used as a physiological stress measure in irrelevant speech studies.

The impact of a speaker's voice quality on auditory perception and cognition: a behavioral and subjective approach.

Introduction: Our voice is key for conveying information and knowledge to others during verbal communication. However, those who heavily depend on their voice, such as teachers and university professors, often develop voice problems, signaled by hoarseness. The aim of this study was to investigate the effect of hoarseness on listeners' memory for auditory-verbal information, listening effort, and listening impression. Methods: Forty-eight normally hearing adults performed two memory tasks that were auditorily presented in varied voice quality (typical vs. hoarse). The tasks were Heard Text Recall, as part of a dual-task paradigm, and auditory Verbal Serial Recall (aVSR). Participants also completed a listening impression questionnaire for both voice qualities. Behavioral measures of memory for auditory-verbal information and listening effort were performance and response time. Subjective measures of listening effort and other aspects of listening impression were questionnaire rating scores. Results: Results showed that, except for the aVSR, behavioral outcomes did not vary with the speaker's voice quality. Regarding the aVSR, we found a significant interaction between voice quality and trial, indicating that participants' recall performance dropped in the beginning of the task in the hoarse-voice condition but not in the typical-voice condition, and then increased again toward the end. Results from the listening impression questionnaire showed that listening to the hoarse voice resulted in significantly increased perceived listening effort, greater annoyance and poorer self-reported performance. Discussion: These findings suggest that hoarseness can, at least subjectively, compromise effective listening. Vocal health may be particularly important in the educational context, where listening and learning are closely linked.

Assemble it like this! - Is AR- or VR-based training an effective alternative to video-based training in manual assembly?

AR- and VR-based training is increasingly being used in the industry to train workers safely and effectively for new tasks. In this study, we investigated and compared the effects of AR-, VR- and video-based training on short- and long-term objective performance measures and subjective evaluations in a manual assembly task. Our results showed that there was no difference between AR-, VR- and video-based training concerning the objective performance measures task completion time and error count. However, in the subjective evaluations VR-based training showed a significantly higher perceived task load and a lower usability rating than the AR- and video-based training regimes. An exploratory analysis additionally revealed partially better results for AR than for VR after adjusting the data for the age of the participants. Future research should further investigate the advantage of AR- and video-based methods over VR when the age and technology experience of participants are taken into account.

Irrelevant speech impairs serial recall of verbal but not spatial items in children and adults.

Immediate serial recall of visually presented items is reliably impaired by task-irrelevant speech that the participants are instructed to ignore ("irrelevant speech effect," ISE). The ISE is stronger with changing speech tokens (words or syllables) when compared to repetitions of single tokens ("changing-state effect," CSE). These phenomena have been attributed to sound-induced diversions of attention away from the focal task (attention capture account), or to specific interference of obligatory, involuntary sound processing with either the integrity of phonological traces in a phonological short-term store (phonological loop account), or the efficiency of a domain-general rehearsal process employed for serial order retention (changing-state account). Aiming to further explore the role of attention, phonological coding, and serial order retention in the ISE, we analyzed the effects of steady-state and changing-state speech on serial order reconstruction of visually presented verbal and spatial items in children (n = 81) and adults (n = 80). In the verbal task, both age groups performed worse with changing-state speech (sequences of different syllables) when compared with steady-state speech (one syllable repeated) and silence. Children were more impaired than adults by both speech sounds. In the spatial task, no disruptive effect of irrelevant speech was found in either group. These results indicate that irrelevant speech evokes similarity-based interference, and thus pose difficulties for the attention-capture and the changing-state account of the ISE.

Differential Effects of Task-Irrelevant Monaural and Binaural Classroom Scenarios on Children's and Adults' Speech Perception, Listening Comprehension, and Visual-Verbal Short-Term Memory.

Most studies investigating the effects of environmental noise on children's cognitive performance examine the impact of monaural noise (i.e., same signal to both ears), oversimplifying multiple aspects of binaural hearing (i.e., adequately reproducing interaural differences and spatial information). In the current study, the effects of a realistic classroom-noise scenario presented either monaurally or binaurally on tasks requiring processing of auditory and visually presented information were analyzed in children and adults. In Experiment 1, across age groups, word identification was more impaired by monaural than by binaural classroom noise, whereas listening comprehension (acting out oral instructions) was equally impaired in both noise conditions. In both tasks, children were more affected than adults. Disturbance ratings were unrelated to the actual performance decrements. Experiment 2 revealed detrimental effects of classroom noise on short-term memory (serial recall of words presented pictorially), which did not differ with age or presentation mode (monaural vs. binaural). The present results add to the evidence for detrimental effects of noise on speech perception and cognitive performance, and their interactions with age, using a realistic classroom-noise scenario. Binaural simulations of real-world auditory environments can improve the external validity of studies on the impact of noise on children's and adults' learning.

Effects of Augmented Reality-, Virtual Reality-, and Mixed Reality-Based Training on Objective Performance Measures and Subjective Evaluations in Manual Assembly Tasks: A Scoping Review.

OBJECTIVE: The present scoping review aims to transform the diverse field of research on the effects of mixed reality-based training on performance in manual assembly tasks into comprehensive statements about industrial needs for and effects of mixed reality-based training. BACKGROUND: Technologies such as augmented and virtual reality, referred to as mixed reality, are seen as promising media for training manual assembly tasks. Nevertheless, current literature shows partly contradictory results, which is due to the diversity of the hardware used, manual assembly tasks as well as methodological approaches to investigate the effects of mixed reality-based training. METHOD: Following the methodological approach of a scoping review, we selected 24 articles according to predefined criteria and analyzed them concerning five key aspects: (1) the needs in the industry for mixed reality-based training, (2) the actual use and classification of mixed reality technologies, (3) defined measures for evaluating the outcomes of mixed reality-based training, (4) findings on objectively measured performance and subjective evaluations, as well as (5) identified research gaps. RESULTS: Regarding the improvement of performance and effectiveness through mixed reality-based training, promising results were found particularly for augmented reality-based training, while virtual reality-based training is mostly-but not consistently-as good as traditional training. APPLICATION: Mixed reality-based training is still not consistently better, but mostly at least as good as traditional training. However, depending on the use case and technology used, the training outcomes in terms of assembly performance and subjective evaluations show promising results of mixed reality-based training.

Does the Macro-Temporal Pattern of Road Traffic Noise Affect Noise Annoyance and Cognitive Performance?

Noise annoyance is usually estimated based on time-averaged noise metrics. However, such metrics ignore other potentially important acoustic characteristics, in particular the macro-temporal pattern of sounds as constituted by quiet periods (noise breaks). Little is known to date about its effect on noise annoyance and cognitive performance, e.g., during work. This study investigated how the macro-temporal pattern of road traffic noise affects short-term noise annoyance and cognitive performance in an attention-based task. In two laboratory experiments, participants worked on the Stroop task, in which performance relies predominantly on attentional functions, while being exposed to different road traffic noise scenarios. These were systematically varied in macro-temporal pattern regarding break duration and distribution (regular, irregular), and played back with moderate LAeq of 42-45 dB(A). Noise annoyance ratings were collected after each scenario. Annoyance was found to vary with the macro-temporal pattern: It decreased with increasing total duration of quiet periods. Further, shorter but more regular breaks were somewhat less annoying than longer but irregular breaks. Since Stroop task performance did not systematically vary with different noise scenarios, differences in annoyance are not moderated by experiencing worsened performance but can be attributed to differences in the macro-temporal pattern of road traffic noise.

The role of spatial separation of two talkers' auditory stimuli in the listener's memory of running speech: listening effort in a non-noisy conversational setting.

OBJECTIVE: To investigate the role of the spatial position of conversing talkers, that is, spatially separated or co-located, in the listener's short-term memory of running speech and listening effort. DESIGN: In two experiments (between-subject), participants underwent a dual-task paradigm, including a listening (primary) task wherein male and female talkers spoke coherent texts. Talkers were either spatially separated or co-located (within-subject). As a secondary task, visually presented tasks were used. Experiment I involved a number-judgement task, and Experiment II entailed switching between number and letter-judgement task. STUDY SAMPLE: Twenty-four young adults who reported normal hearing and normal or corrected to normal vision participated in each experiment. They were all students from the RWTH Aachen University. RESULTS: In both experiments, similar short-term memory performance of running speech was found independently of talkers being spatially separated or co-located. Performance in the secondary tasks, however, differed between these two talkers' auditory stimuli conditions, indicating that spatially separated talkers imposed reduced listening effort compared to their co-location. CONCLUSION: The findings indicated that auditory-perceptive information, such as the spatial position of talkers, plays a role in higher-level auditory cognition, that is, short-term memory of running speech, even when listening in quiet.

Character Customization With Cosmetic Microtransactions in Games: Subjective Experience and Objective Performance.

Free games that are monetized by selling virtual items, such as cosmetic microtransactions for one's avatar, seem to offer a better gaming experience to paying players. To experimentally explore this phenomenon, the effects of character customization with cosmetic microtransactions on objective and self-estimated player performance, subjective identification with the avatar, fun and the players' perceived competence were examined in the game League of Legends. This study introduces a new laboratory-based, experimental task to objectively measure within-game player performance. Each participant performed this game-based task in two different conditions: With a character that was customized using a provided set of cosmetic microtransactions and with a default character. Results showed that customization increased subjective identification with the player character. However, objective performance measures were unaffected by this manipulation although the novel experimental approach provided reliable performance results. Additionally, identification was positively related to perceived competence, fun, and self-estimated performance. Implications for the design of cosmetic microtransactions and their influence on competitive gaming are discussed.

Audio-video virtual reality environments in building acoustics: An exemplary study reproducing performance results and subjective ratings of a laboratory listening experiment.

The present study adopted a human-centred approach to explore the potential of audio-video Virtual Reality (VR) to evaluate indoor noise protection by building characteristics. Different background speech conditions, convolved with sound insulation filters of adjacent office rooms, were presented in a VR office environment and the effects on cognitive performances and subjective ratings were measured. The found effect patterns were the same as those obtained in a real laboratory setting reported by Schlittmeier, Hellbrück, Thaden, and Vorländer. [(2008). Ergonomics 51, 719-736]. This exemplary study promises various options for research on noise effects by the use of virtual built environments which are of high plausibility and unlimited variability.

Effects of Different Spectral Shapes and Amplitude Modulation of Broadband Noise on Annoyance Reactions in a Controlled Listening Experiment.

Environmental noise from transportation or industrial infrastructure typically has a broad frequency range. Different sources may have disparate acoustical characteristics, which may in turn affect noise annoyance. However, knowledge of the relative contribution of the different acoustical characteristics of broadband noise to annoyance is still scarce. In this study, the subjectively perceived short-term (acute) annoyance reactions to different broadband sounds (namely, realistic outdoor wind turbine and artificial, generic sounds) at 40 dBA were investigated in a controlled laboratory listening experiment. Combined with the factorial design of the experiment, the sounds allowed for separation of the effects of three acoustical characteristics on annoyance, namely, spectral shape, depth of periodic amplitude modulation (AM), and occurrence (or absence) of random AM. Fifty-two participants rated their annoyance with the sounds. Annoyance increased with increasing energy content in the low-frequency range as well as with depth of periodic AM, and was higher in situations with random AM than without. Similar annoyance changes would be evoked by sound pressure level changes of up to 8 dB. The results suggest that besides standard sound pressure level metrics, other acoustical characteristics of (broadband) noise should also be considered in environmental impact assessments, e.g., in the context of wind turbine installations.

Short-term annoyance reactions to stationary and time-varying wind turbine and road traffic noise: A laboratory study.

Current literature suggests that wind turbine noise is more annoying than transportation noise. To date, however, it is not known which acoustic characteristics of wind turbines alone, i.e., without effect modifiers such as visibility, are associated with annoyance. The objective of this study was therefore to investigate and compare the short-term noise annoyance reactions to wind turbines and road traffic in controlled laboratory listening tests. A set of acoustic scenarios was created which, combined with the factorial design of the listening tests, allowed separating the individual associations of three acoustic characteristics with annoyance, namely, source type (wind turbine, road traffic), A-weighted sound pressure level, and amplitude modulation (without, periodic, random). Sixty participants rated their annoyance to the sounds. At the same A-weighted sound pressure level, wind turbine noise was found to be associated with higher annoyance than road traffic noise, particularly with amplitude modulation. The increased annoyance to amplitude modulation of wind turbines is not related to its periodicity, but seems to depend on the modulation frequency range. The study discloses a direct link of different acoustic characteristics to annoyance, yet the generalizability to long-term exposure in the field still needs to be verified.

Unmasking the effects of masking on performance: The potential of multiple-voice masking in the office environment.

Broadband noise is often used as a masking sound to combat the negative consequences of background speech on performance in open-plan offices. As office workers generally dislike broadband noise, it is important to find alternatives that are more appreciated while being at least not less effective. The purpose of experiment 1 was to compare broadband noise with two alternatives-multiple voices and water waves-in the context of a serial short-term memory task. A single voice impaired memory in comparison with silence, but when the single voice was masked with multiple voices, performance was on level with silence. Experiment 2 explored the benefits of multiple-voice masking in more detail (by comparing one voice, three voices, five voices, and seven voices) in the context of word processed writing (arguably a more office-relevant task). Performance (i.e., writing fluency) increased linearly from worst performance in the one-voice condition to best performance in the seven-voice condition. Psychological mechanisms underpinning these effects are discussed.

The effects of spectral and temporal parameters on perceived confirmation of an auditory non-speech signal.

In human-machine interactions, the confirmation of an action or input is a very important information for users. A paired comparison experiment explored the effects of four acoustic parameters on the perceived confirmation of auditory non-speech signals. Reducing the frequency-ratio and the pulse-to-pulse time between two successive pulses increased perceived confirmation. The effects of the parameters frequency and number of pulses were not clear-cut. The results provide information for designing auditory confirmation signals. It is shown that findings about the effects of certain parameters on the perceived urgency of warning signals cannot be easily inverted to perceived confirmation.

The impact of road traffic noise on cognitive performance in attention-based tasks depends on noise level even within moderate-level ranges.

Little empirical evidence is available regarding the effects of road traffic noise on cognitive performance in adults, although traffic noise can be heard at many offices and home office workplaces. Our study tested the impact of road traffic noise at different levels (50 dB(A), 60 dB(A), 70 dB(A)) on performance in three tasks that differed with respect to their dependency on attentional and storage functions, as follows: The Stroop task, in which performance relied predominantly on attentional functions (e.g., inhibition of automated responses; Experiment 1: n = 24); a non-automated multistage mental arithmetic task calling for both attentional and storage functions (Exp. 2: n = 18); and verbal serial recall, which placed a burden predominantly on storage functions (Experiment 3: n = 18). Better performance was observed during moderate road traffic noise at 50 dB(A) compared to loud traffic noise at 70 dB(A) in attention-based tasks (Experiments 1-2). This contrasted with the effects of irrelevant speech (60 dB(A)), which was included in the experiments as a well-explored and common noise source in office settings. A disturbance impact of background speech was only given in the two tasks that called for storage functions (Experiments 2-3). In addition to the performance data, subjective annoyance ratings were collected. Consistent with the level effect of road traffic noise found in the performance data, a moderate road traffic noise at 50 dB(A) was perceived as significantly less annoying than a loud road traffic noise at 70 dB(A), which was found, however, independently of the task at hand. Furthermore, the background sound condition with the highest detrimental performance effect in a task was also rated as most annoying in this task, i.e., traffic noise at 70 dB(A) in the Stroop task, and background speech in the mental arithmetic and serial recall tasks.

Combined effects of acoustic and visual distraction on cognitive performance and well-being.

Information work is usually performed in offices and influenced by the combined effects of acoustics, room climate, lighting and air quality. However, the principal part of literature solely focuses on the individual effects of physical parameters. This study (n = 32) investigates the combined effects of acoustic and visual distraction with regard to cognitive performance and well-being. Therefore low level background speech (40 dB(A)) of good or poor intelligibility was combined with either static or dynamic lighting. Experimental testing lasted for approx. 7 h for each participant and was conducted in mock-up offices. No interaction effects of background speech and lighting conditions with regard to cognitive performance were found. However, the results prove that even low level background speech of high intelligibility significantly impairs short-term memory, reasoning ability and well-being. But no effect of background speech on text comprehension and sustained attention was found. Visual distraction due to dynamic lighting caused significant complaints but did not impair performance. An interaction effect of background speech and lighting conditions was found with regard to perceived performance during task processing. Test persons only felt to perform better, if background speech of low intelligibility was combined with static lighting. It is shown that the effects on cognitive performance and well-being must be considered separately since these effects are rarely consistent.

Algorithmic modeling of the irrelevant sound effect (ISE) by the hearing sensation fluctuation strength.

Background sounds, such as narration, music with prominent staccato passages, and office noise impair verbal short-term memory even when these sounds are irrelevant. This irrelevant sound effect (ISE) is evoked by so-called changing-state sounds that are characterized by a distinct temporal structure with varying successive auditory-perceptive tokens. However, because of the absence of an appropriate psychoacoustically based instrumental measure, the disturbing impact of a given speech or nonspeech sound could not be predicted until now, but necessitated behavioral testing. Our database for parametric modeling of the ISE included approximately 40 background sounds (e.g., speech, music, tone sequences, office noise, traffic noise) and corresponding performance data that was collected from 70 behavioral measurements of verbal short-term memory. The hearing sensation fluctuation strength was chosen to model the ISE and describes the percept of fluctuations when listening to slowly modulated sounds (f(mod) < 20 Hz). On the basis of the fluctuation strength of background sounds, the algorithm estimated behavioral performance data in 63 of 70 cases within the interquartile ranges. In particular, all real-world sounds were modeled adequately, whereas the algorithm overestimated the (non-)disturbance impact of synthetic steady-state sounds that were constituted by a repeated vowel or tone. Implications of the algorithm's strengths and prediction errors are discussed.

What characterizes changing-state speech in affecting short-term memory? An EEG study on the irrelevant sound effect.

The irrelevant sound effect (ISE) describes reduced verbal short-term memory during irrelevant changing-state sounds which consist of different and distinct auditory tokens. Steady-state sounds lack such changing-state features and do not impair performance. An EEG experiment (N=16) explored the distinguishing neurophysiological aspects of detrimental changing-state speech (3-token sequence) compared to ineffective steady-state speech (1-token sequence) on serial recall performance. We analyzed evoked and induced activity related to the memory items as well as spectral activity during the retention phase. The main finding is that the behavioral sound effect was exclusively reflected by attenuated token-induced gamma activation most pronounced between 50-60 Hz and 50-100 ms post-stimulus onset. Changing-state speech seems to disrupt a behaviorally relevant ongoing process during target presentation (e.g., the serial binding of the items).