Your browser doesn't support javascript.
Masking of the mouth area impairs reconstruction of acoustic speech features and higher-level segmentational features in the presence of a distractor speaker.
Haider, Chandra Leon; Suess, Nina; Hauswald, Anne; Park, Hyojin; Weisz, Nathan.
  • Haider CL; Centre for Cognitive Neuroscience and Department of Psychology, University of Salzburg, Austria. Electronic address: chandraleon.haider@plus.ac.at.
  • Suess N; Centre for Cognitive Neuroscience and Department of Psychology, University of Salzburg, Austria.
  • Hauswald A; Centre for Cognitive Neuroscience and Department of Psychology, University of Salzburg, Austria.
  • Park H; School of Psychology & Centre for Human Brain Health (CHBH), University of Birmingham, Birmingham, UK.
  • Weisz N; Centre for Cognitive Neuroscience and Department of Psychology, University of Salzburg, Austria; Neuroscience Institute, Christian Doppler University Hospital, Paracelsus Medical University, Salzburg, Austria.
Neuroimage ; 252: 119044, 2022 05 15.
Article in English | MEDLINE | ID: covidwho-1756286
ABSTRACT
Multisensory integration enables stimulus representation even when the sensory input in a single modality is weak. In the context of speech, when confronted with a degraded acoustic signal, congruent visual inputs promote comprehension. When this input is masked, speech comprehension consequently becomes more difficult. But it still remains inconclusive which levels of speech processing are affected under which circumstances by occluding the mouth area. To answer this question, we conducted an audiovisual (AV) multi-speaker experiment using naturalistic speech. In half of the trials, the target speaker wore a (surgical) face mask, while we measured the brain activity of normal hearing participants via magnetoencephalography (MEG). We additionally added a distractor speaker in half of the trials in order to create an ecologically difficult listening situation. A decoding model on the clear AV speech was trained and used to reconstruct crucial speech features in each condition. We found significant main effects of face masks on the reconstruction of acoustic features, such as the speech envelope and spectral speech features (i.e. pitch and formant frequencies), while reconstruction of higher level features of speech segmentation (phoneme and word onsets) were especially impaired through masks in difficult listening situations. As we used surgical face masks in our study, which only show mild effects on speech acoustics, we interpret our findings as the result of the missing visual input. Our findings extend previous behavioural results, by demonstrating the complex contextual effects of occluding relevant visual information on speech processing.
Subject(s)
Keywords

Full text: Available Collection: International databases Database: MEDLINE Main subject: Speech / Speech Perception Limits: Humans Language: English Journal: Neuroimage Journal subject: Diagnostic Imaging Year: 2022 Document Type: Article

Similar

MEDLINE

...
LILACS

LIS


Full text: Available Collection: International databases Database: MEDLINE Main subject: Speech / Speech Perception Limits: Humans Language: English Journal: Neuroimage Journal subject: Diagnostic Imaging Year: 2022 Document Type: Article