Interrupted mosaic speech revisited: Gain and loss in intelligibility by stretchinga).

Our previous investigation on the effect of stretching spectrotemporally degraded and temporally interrupted speech stimuli showed remarkable intelligibility gains [Udea, Takeichi, and Wakamiya (2022). J. Acoust. Soc. Am. 152(2), 970-980]. In this previous study, however, gap durations and temporal resolution were confounded. In the current investigation, we therefore observed the intelligibility of so-called mosaic speech while dissociating the effects of interruption and temporal resolution. The intelligibility of mosaic speech (20 frequency bands and 20 ms segment duration) declined from 95% to 78% and 33% by interrupting it with 20 and 80 ms gaps. Intelligibility improved, however, to 92% and 54% (14% and 21% gains for 20 and 80 ms gaps, respectively) by stretching mosaic segments to fill silent gaps (n = 21). By contrast, the intelligibility was impoverished to a minimum of 9% (7% loss) when stretching stimuli interrupted with 160 ms gaps. Explanations based on auditory grouping, modulation unmasking, or phonemic restoration may account for the intelligibility improvement by stretching, but not for the loss. The probability summation model accounted for "U"-shaped intelligibility curves and the gain and loss of intelligibility, suggesting that perceptual unit length and speech rate may affect the intelligibility of spectrotemporally degraded speech stimuli.

Checkerboard and interrupted speech: Intelligibility contrasts related to factor-analysis-based frequency bandsa).

It has been shown that the intelligibility of checkerboard speech stimuli, in which speech signals were periodically interrupted in time and frequency, drastically varied according to the combination of the number of frequency bands (2-20) and segment duration (20-320 ms). However, the effects of the number of frequency bands between 4 and 20 and the frequency division parameters on intelligibility have been largely unknown. Here, we show that speech intelligibility was lowest in four-band checkerboard speech stimuli, except for the 320-ms segment duration. Then, temporally interrupted speech stimuli and eight-band checkerboard speech stimuli came in this order (N = 19 and 20). At the same time, U-shaped intelligibility curves were observed for four-band and possibly eight-band checkerboard speech stimuli. Furthermore, different parameters of frequency division resulted in small but significant intelligibility differences at the 160- and 320-ms segment duration in four-band checkerboard speech stimuli. These results suggest that factor-analysis-based four frequency bands, representing groups of critical bands correlating with each other in speech power fluctuations, work as speech cue channels essential for speech perception. Moreover, a probability summation model for perceptual units, consisting of a sub-unit process and a supra-unit process that receives outputs of the speech cue channels, may account for the U-shaped intelligibility curves.

Visual and haptic cues in processing occlusion.

Introduction: Although shape is effective in processing occlusion, ambiguities in segmentation can also be addressed using depth discontinuity given visually and haptically. This study elucidates the contribution of visual and haptic cues to depth discontinuity in processing occlusion. Methods: A virtual reality experiment was conducted with 15 students as participants. Word stimuli were presented on a head-mounted display for recognition. The central part of the words was masked with a virtual ribbon placed at different depths so that the ribbon appeared as an occlusion. The visual depth cue was either present with binocular stereopsis or absent with monocular presentation. The haptic cue was either missing, provided consecutively, or concurrently, by actively tracing a real off-screen bar edge that was positionally aligned with the ribbon in the virtual space. Recognition performance was compared between depth cue conditions. Results: We found that word recognition was better with the stereoscopic cue but not with the haptic cue, although both cues contributed to greater confidence in depth estimation. The performance was better when the ribbon was at the farther depth plane to appear as a hollow, rather than when it was at the nearer depth plane to cover the word. Discussion: The results indicate that occlusion is processed in the human brain by visual input only despite the apparent effectiveness of haptic space perception, reflecting a complex set of natural constraints.

Auditory grouping is necessary to understand interrupted mosaic speech stimuli.

The intelligibility of interrupted speech stimuli has been known to be almost perfect when segment duration is shorter than 80 ms, which means that the interrupted segments are perceptually organized into a coherent stream under this condition. However, why listeners can successfully group the interrupted segments into a coherent stream has been largely unknown. Here, we show that the intelligibility for mosaic speech in which original speech was segmented in frequency and time and noise-vocoded with the average power in each unit was largely reduced by periodical interruption. At the same time, the intelligibility could be recovered by promoting auditory grouping of the interrupted segments by stretching the segments up to 40 ms and reducing the gaps, provided that the number of frequency bands was enough ( ≥ 4) and the original segment duration was equal to or less than 40 ms. The interruption was devastating for mosaic speech stimuli, very likely because the deprivation of periodicity and temporal fine structure with mosaicking prevented successful auditory grouping for the interrupted segments.

Detection of deviance in Japanese kanji compound words.

Reading fluency is based on the automatic visual recognition of words. As a manifestation of the automatic processing of words, an automatic deviance detection of visual word stimuli can be observed in the early stages of visual recognition. To clarify whether this phenomenon occurs with Japanese kanji compounds-since their lexicality is related to semantic association-we investigated the brain response by utilizing three types of deviants: differences in font type, lexically correct or incorrect Japanese kanji compound words and pseudo-kanji characters modified from correct and incorrect compounds. We employed magnetoencephalography (MEG) to evaluate the spatiotemporal profiles of the related brain regions. The study included 22 adult native Japanese speakers (16 females). The abovementioned three kinds of stimuli containing 20% deviants were presented during the MEG measurement. Activity in the occipital pole region of the brain was observed upon the detection of font-type deviance within 250 ms of stimulus onset. Although no significant activity upon detecting lexically correct/incorrect kanji compounds or pseudo-kanji character deviations was observed, the activity in the posterior transverse region of the collateral sulcus (pCoS)-which is a fusiform neighboring area-was larger when detecting lexically correct kanji compounds than when detecting pseudo-kanji characters. Taken together, these results support the notion that the automatic detection of deviance in kanji compounds may be limited to a low-level feature, such as the stimulus stroke thickness.

The common limitations in auditory temporal processing for Mandarin Chinese and Japanese.

The present investigation focused on how temporal degradation affected intelligibility in two types of languages, i.e., a tonal language (Mandarin Chinese) and a non-tonal language (Japanese). The temporal resolution of common daily-life sentences spoken by native speakers was systematically degraded with mosaicking (mosaicising), in which the power of original speech in each of regularly spaced time-frequency unit was averaged and temporal fine structure was removed. The results showed very similar patterns of variations in intelligibility for these two languages over a wide range of temporal resolution, implying that temporal degradation crucially affected speech cues other than tonal cues in degraded speech without temporal fine structure. Specifically, the intelligibility of both languages maintained a ceiling up to about the 40-ms segment duration, then the performance gradually declined with increasing segment duration, and reached a floor at about the 150-ms segment duration or longer. The same limitations for the ceiling performance up to 40 ms appeared for the other method of degradation, i.e., local time-reversal, implying that a common temporal processing mechanism was related to the limitations. The general tendency fitted to a dual time-window model of speech processing, in which a short (~ 20-30 ms) and a long (~ 200 ms) time-window run in parallel.

Improvement of brain function after surgery in infants with posterior quadrant cortical dysplasia.

OBJECTIVE: To reveal whether neurodevelopmental outcome of infants after epilepsy surgery can be quantitatively assessed by electroencephalography (EEG) functional connectivity analysis. METHODS: We enrolled 13 infants with posterior quadrant dysplasia aged <2 years who were treated using posterior quadrantectomy and 21 age-matched infants. EEG was performed both before and one year after surgery. Developmental quotient (DQ) was assessed both before and 3 years after surgery. The phase lag index (PLI) of three different pairs of electrodes in the nonsurgical hemisphere, i.e., the anterior short distance (ASD), posterior short distance (PSD), and long distance (LD) pairs, were calculated as indices of brain connectivity. The relationship between the PLI and DQ was evaluated. RESULTS: Overall, 77% infants experienced seizure freedom after surgery. The beta- and gamma- range PLI of PSD pairs increased preoperatively. All these pairs normalized postoperatively. Simple linear regression analysis revealed a significant relationship between the postoperative DQ and the postoperative beta-band PLI of ASD pairs. CONCLUSION: Preoperative abnormal hyper-connectivity was normalized to the control level after surgery. The postoperative hyperconnectivity was associated with long-term neurodevelopmental improvement. SIGNIFICANCE: PLI quantifies neurodevelopmental improvements after posterior quadrantectomy.

Checkerboard speech vs interrupted speech: Effects of spectrotemporal segmentation on intelligibility.

The intelligibility of interrupted speech (interrupted over time) and checkerboard speech (interrupted over time-by-frequency), both of which retained a half of the original speech, was examined. The intelligibility of interrupted speech stimuli decreased as segment duration increased. 20-band checkerboard speech stimuli brought nearly 100% intelligibility irrespective of segment duration, whereas, with 2 and 4 frequency bands, a trough of 35%-40% appeared at the 160-ms segment duration. Mosaic speech stimuli (power was averaged over a time-frequency unit) yielded generally poor intelligibility ( ⩽10%). The results revealed the limitations of underlying auditory organization for speech cues scattered in a time-frequency domain.

Visualization by P-flow: gradient- and feature-based optical flow and vector fields extracted from image analysis.

We proposed a method for extracting the optical flow suitable for visualization, pseudo-flow (P-flow), from a natural movie [Exp. Brain Res.237, 3321 (2019)EXBRAP0014-481910.1007/s00221-019-05674-0]. The P-flow algorithm comprises two stages: (1) extraction of a local motion vector field from two successive frames and (2) tracking of vectors between two successive frame pairs. In this study, we show that while P-flow takes a feature (vector) tracking approach, it is also classified as a gradient-based approach that satisfies the brightness constancy constraint. We also incorporate interpolation and a corner detector to address the shortcomings associated with the two approaches.

Association between lack of functional connectivity of the frontal brain region and poor response inhibition in children with frontal lobe epilepsy.

PURPOSE: We investigated the relationship between electroencephalographic (EEG) functional connectivity and executive function in children with frontal lobe epilepsy (FLE). METHODS: We enrolled 24 children with FLE (mean age, 11.0 years; 13 boys) and 22 sex-, age-, and intelligence-matched typically developing children (TDC) to undergo 19-channel EEG during light sleep. We estimated functional connectivity using the phase lag index (PLI) that captures the synchronization of EEG. We also performed continuous performance tests (CPTs) on the children and obtained questionnaire responses on attention deficit hyperactivity disorder and oppositional defiant disorder (ODD). RESULTS: The average gamma PLI was lower in the FLE group than in the TDC group, especially between long-distance frontoparietal pairs, between interhemispheric frontal pairs, and between interhemispheric parietotemporal pairs. Gamma PLIs with long-distance frontoparietal and interhemispheric frontal pairs were positively associated with inattention, ODD scores, omission error, and reaction time in the FLE group but not in the TDC group. Conversely, they were negatively associated with age, hyperactivity score, and commission error. CONCLUSIONS: A lack of functional connectivity of the frontal brain regions in children with FLE was associated with poor response inhibition.

Atypical gamma functional connectivity pattern during light sleep in children with attention deficit hyperactivity disorder.

INTRODUCTION: We examined functional connectivity analyses in electroencephalograms (EEGs) of patients with attention deficit hyperactivitydisorder (ADHD) and in those of typically developing children (TDC) to uncover neurobiological abnormalities. METHODOLOGY: We enrolled 31 children with ADHD (mean age 11.1 years; 23 boys) and 17 sex-, age-, and intelligence-matched TDC to undergo 19-channel EEGs during light sleep. We estimated functional connectivity using the phase lag index (PLI) and coherence measurements that capture the synchronization of EEG signals and graphed metrics with GRETNA. We also performed continuous performance tests (CPTs) on the children and obtained answered questionnaires on ADHD and autism spectrum disorder. RESULTS: The central-to-posterior gamma PLI was lower in children with ADHD than that in TDC. The other PLI frequency bands and all coherence frequency bands were not statistically different between both groups. Individuals with high hyperactivity scores on questionnaires and low reaction times (SDs) on CPT had low motor and occipital pairs of gamma PLIs. Graph metrics showed no differences between the groups. CONCLUSIONS: The difference in averaged gamma PLI (especially with motor and occipital pairs) between groups was more suitable for diagnosis than the averaged coherence. Lower averaged gamma PLIs reflected more severe ADHD symptoms. A prospective study with more controlled conditions is warranted to determine if gamma-band PLI can be used as an auxiliary tool for ADHD diagnosis.

Alteration of the anatomical covariance network after corpus callosotomy in pediatric intractable epilepsy.

PURPOSE: This study aimed to use graph theoretical analysis of anatomical covariance derived from structural MRI to reveal how the gray matter connectivity pattern is altered after corpus callosotomy (CC). MATERIALS AND METHODS: We recruited 21 patients with epilepsy who had undergone CC. Enrollment criteria were applied: (1) no lesion identified on brain MRI; (2) no history of other brain surgery; and (3) age not younger than 3 years and not older than 18 years at preoperative MRI evaluation. The most common epilepsy syndrome was Lennox-Gastaut syndrome (11 patients). For voxel-based morphometry, the normalized gray matter images of pre-CC and post-CC patients were analyzed with SPM12 (voxel-level threshold of p<0.05 [familywise error-corrected]). Secondly, the images of both groups were subjected to graph theoretical analysis using the Graph Analysis Toolbox with SPM8. Each group was also compared with 32 age- and sex-matched control patients without brain diseases. RESULTS: Comparisons between the pre- and post-CC groups revealed a significant reduction in seizure frequency with no change in mean intelligence quotient/developmental quotient levels. There was no relationship among the three groups in global network metrics or in targeted attack. A regional comparison of betweenness centrality revealed decreased connectivity to and from the right middle cingulate gyri and medial side of the right superior frontal gyrus and a partial shift in the distribution of betweenness centrality hubs to the normal location. Significantly lower resilience to random failure was found after versus before CC and versus controls (p = 0.0450 and p = 0.0200, respectively). CONCLUSION: Graph theoretical analysis of anatomical covariance derived from structural imaging revealed two neural network effects of resection associated with seizure reduction: the reappearance of a structural network comparable to that in healthy children and reduced connectivity along the median line, including the middle cingulate gyrus.

Vection induced by low-level motion extracted from complex animation films.

This study examined the contributions of low-, mid- and high-level visual motion information to vection. We compared the vection experiences induced by hand-drawn and computer-generated animation clips to those induced by versions of these movies that contained only their pure optic flow. While the original movies were found to induce longer and stronger vection experiences than the pure optic flow, vection onsets were not significantly altered by removing the mid- and high-level information. We conclude that low-level visual motion information appears to be important for vection induction, whereas mid- and higher-level display information appears to be important for sustaining and strengthening this vection after its initial induction.

Spatiotemporal brain dynamics of auditory temporal assimilation.

Time is a fundamental dimension, but millisecond-level judgments sometimes lead to perceptual illusions. We previously introduced a "time-shrinking illusion" using a psychological paradigm that induces auditory temporal assimilation (ATA). In ATA, the duration of two successive intervals (T1 and T2), marked by three auditory stimuli, can be perceived as equal when they are not. Here, we investigate the spatiotemporal profile of human temporal judgments using magnetoencephalography (MEG). Behavioural results showed typical ATA: participants judged T1 and T2 as equal when T2 - T1 ≤ +80 ms. MEG source-localisation analysis demonstrated that regional activity differences between judgment and no-judgment conditions emerged in the temporoparietal junction (TPJ) during T2. This observation in the TPJ may indicate its involvement in the encoding process when T1 ≠ T2. Activation in the inferior frontal gyrus (IFG) was enhanced irrespective of the stimulus patterns when participants engaged in temporal judgment. Furthermore, just after the final marker, activity in the IFG was enhanced specifically for the time-shrinking pattern. This indicates that activity in the IFG is also related to the illusory perception of time-interval equality. Based on these observations, we propose neural signatures for judgments of temporal equality in the human brain.

Novel method of extracting motion from natural movies.

BACKGROUND: The visual system in primates can be segregated into motion and shape pathways. Interaction occurs at multiple stages along these pathways. Processing of shape-from-motion and biological motion is considered to be a higher-order integration process involving motion and shape information. However, relatively limited types of stimuli have been used in previous studies on these integration processes. NEW METHOD: We propose a new algorithm to extract object motion information from natural movies and to move random dots in accordance with the information. The object motion information is extracted by estimating the dynamics of local normal vectors of the image intensity projected onto the x-y plane of the movie. RESULTS: An electrophysiological experiment on two adult common marmoset monkeys (Callithrix jacchus) showed that the natural and random dot movies generated with this new algorithm yielded comparable neural responses in the middle temporal visual area. COMPARISON WITH EXISTING METHODS: In principle, this algorithm provided random dot motion stimuli containing shape information for arbitrary natural movies. This new method is expected to expand the neurophysiological and psychophysical experimental protocols to elucidate the integration processing of motion and shape information in biological systems. CONCLUSIONS: The novel algorithm proposed here was effective in extracting object motion information from natural movies and provided new motion stimuli to investigate higher-order motion information processing.

Perceptual inequality between two neighboring time intervals defined by sound markers: correspondence between neurophysiological and psychological data.

Brain activity related to time estimation processes in humans was analyzed using a perceptual phenomenon called auditory temporal assimilation. In a typical stimulus condition, two neighboring time intervals (T1 and T2 in this order) are perceived as equal even when the physical lengths of these time intervals are considerably different. Our previous event-related potential (ERP) study demonstrated that a slow negative component (SNCt) appears in the right-frontal brain area (around the F8 electrode) after T2, which is associated with judgment of the equality/inequality of T1 and T2. In the present study, we conducted two ERP experiments to further confirm the robustness of the SNCt. The stimulus patterns consisted of two neighboring time intervals marked by three successive tone bursts. Thirteen participants only listened to the patterns in the first session, and judged the equality/inequality of T1 and T2 in the next session. Behavioral data showed typical temporal assimilation. The ERP data revealed that three components (N1; contingent negative variation, CNV; and SNCt) emerged related to the temporal judgment. The N1 appeared in the central area, and its peak latencies corresponded to the physical timing of each marker onset. The CNV component appeared in the frontal area during T2 presentation, and its amplitude increased as a function of T1. The SNCt appeared in the right-frontal area after the presentation of T1 and T2, and its magnitude was larger for the temporal patterns causing perceptual inequality. The SNCt was also correlated with the perceptual equality/inequality of the same stimulus pattern, and continued up to about 400 ms after the end of T2. These results suggest that the SNCt can be a signature of equality/inequality judgment, which derives from the comparison of the two neighboring time intervals.

Audio-vocal monitoring system revealed by mu-rhythm activity.

Understanding the neural mechanisms underlying speech production has a number of potential practical applications. Speech production involves multiple feedback loops. An audio-vocal monitoring system plays an important role in speech production, based on auditory feedback about the speaker's own voice. Here we investigated the mu-rhythm activity associated with speech production by examining event-related desynchronization and synchronization in conditions of delayed auditory feedback (DAF) and noise feedback (Lombard). In Experiment 1, we confirmed that the mu-rhythms were detectable for a conventional finger-tapping task, and vocalization. In Experiment 2, we examined the mu-rhythms for imagined speech production. We tested whether the same motor-related mu-rhythm activity was exhibited while participants listened to their own voice, and while reading. The mu-rhythms were observed for overt vocalization and covert reading, while listening to simulated auditory feedback of the participants' own voice reading text. In addition, we found that the mu-rhythm associated with listening was boosted and attenuated under the DAF and Lombard conditions, respectively. This is consistent with the notion that auditory feedback is important for the audio-vocal monitoring system in speech production. This paradigm may help clarify the way in which auditory feedback supports motor planning, as indexed by the motor-related mu-rhythm.

An electroencephalographic investigation of the filled-duration illusion.

The study investigated how the brain activity changed when participants were engaged in a temporal production task known as the "filled-duration illusion." Twelve right-handed participants were asked to memorize and reproduce the duration of time intervals (600 or 800 ms) bounded by two flashes. Random trials contained auditory stimuli in the form of three 20 ms sounds between the flashes. In one session, the participants were asked to ignore the presence of the sounds, and in the other, they were instructed to pay attention to sounds. The behavioral results showed that duration reproduction was clearly affected by the presence of the sounds and the duration of time intervals. The filled-duration illusion occurred when there were sounds; the participants overestimated the interval in the 600-ms interval condition with sounds. On the other hand, the participants underestimated the 800-ms interval condition without sounds. During the presentation of the interval to be encoded, the contingent negative variation (CNV) appeared around the prefrontal scalp site, and P300 appeared around the parieto-central scalp site. The CNV grew larger when the intervals contained the sounds, whereas the P300 grew larger when the intervals were 800 ms and did not contain the sounds. During the reproduction of the interval to be presented, the Bereitschaftspotential (BP) appeared over the fronto-central scalp site from 1000 ms before the participants' response. The BP could refer to the decision making process associated with the duration reproduction. The occurrence of three event-related potentials (ERPs), the P300, CNV, and BP, suggests that the fronto-parietal area, together with supplementary motor area (SMA), is associated with timing and time perception, and magnitude of these potentials is modulted by the "filled-duration illusion".