Co-designing the integration of voice-based conversational AI and web augmentation to amplify web inclusivity.

The Web has become an essential resource but is not yet accessible to everyone. Assistive technologies and innovative, intelligent frameworks, for example, those using conversational AI, help overcome some exclusions. However, some users still experience barriers. This paper shows how a human-centered approach can shed light on technology limitations and gaps. It reports on a three-step process (focus group, co-design, and preliminary validation) that we adopted to investigate how people with speech impairments, e.g., dysarthria, browse the Web and how barriers can be reduced. The methodology helped us identify challenges and create new solutions, i.e., patterns for Web browsing, by combining voice-based conversational AI, customized for impaired speech, with techniques for the visual augmentation of web pages. While current trends in AI research focus on more and more powerful large models, participants remarked how current conversational systems do not meet their needs, and how it is important to consider each one's specificity for a technology to be called inclusive.

Exploring the effect of lexicality and listener experience on gradient ratings of Swedish sibilant fricatives.

This study explores the influence of lexicality on gradient judgments of Swedish sibilant fricatives by contrasting ratings of initial fricatives in words and word fragments (initial CV-syllables). Visual-Analogue Scale (VAS) judgments were elicited from experienced listeners (speech-language pathologists; SLPs) and inexperienced listeners, and compared with respect to the effects of lexicality using Bayesian mixed-effects beta regression. Overall, SLPs had higher intra- and interrater reliability than inexperienced listeners. SLPs as a group also rated fricatives as more target-like, with higher precision, than did inexperienced listeners. An effect of lexicality was observed for all individual listeners, though the magnitude of the effect varied. Although SLP's ratings of Swedish children's initial voiceless fricatives were less influenced by lexicality, our results indicate that previous findings concerning VAS ratings of non-lexical CV-syllables cannot be directly transferred to the clinical context, without consideration of possible lexical bias.

The potential impact of literacy intervention on speech sound production in students with intellectual disability and communication difficulties.

A small body of research and reports from educational and clinical practice suggest that teaching literacy skills may facilitate the development of speech sound production in students with intellectual disabilities (ID). However, intervention research is needed to test the potential connection. This study aimed to investigate whether twelve weeks of systematic, digital literacy intervention enhanced speech sound production in students with ID and communication difficulties. A sample of 121 students with ID were assigned to four different groups: phonics-based, comprehension-based, a combination with both phonics- and comprehension-based intervention and a comparison group with teaching-as-usual. Speech sound production was assessed before and after the intervention. The results on the data without the imputed variable suggested a significant positive effect of systematic, digital literacy interventions on speech sound production. However, results from sensitivity analyses with imputed missing data was more ambiguous, with the effect only approaching significance (ps = .05-.07) for one of the interventions. Nonetheless, we tentatively suggest that systematic, digital literacy intervention could support speech development in students with ID and communication difficulties. Future research should be done to confirm and further elucidate the functional mechanisms of this link, so that we may have a better understanding and can improve instruction and the pivotal abilities of speech and reading.

Augmentative and alternative communication for individuals with post-stroke aphasia: perspectives of South African speech-language pathologists.

This qualitative study aimed to describe speech-language pathologists' (SLPs') perspectives on augmentative and alternative communication (AAC) use for people with post-stroke aphasia focusing on: (a) current AAC practice, (b) factors that influence the use of AAC, and (c) the success and relevance of AAC interventions. Semi-structured interviews took place with ten South African SLPs with experience in aphasia intervention. The transcribed interviews were thematically analyzed using a six-phase process of inductive and deductive analysis within a phenomenological framework. All the participants use AAC with their clients, employing a variety of approaches that reflect their diverse settings, experiences, and perspectives on AAC. AAC use is complex, and SLPs make conscious choices considering multiple factors. Barriers to use were often associated with limited resources in the low- and middle-income country (LMIC) context, but most participants retained a positive view of AAC, actively working to circumvent barriers to use. Participants consistently emphasized the vital role of partners in communication interactions, linked to the importance of defining AAC broadly. It is necessary to advance the integration of AAC into rehabilitation plans to improve communication and social participation outcomes for people with post-stroke aphasia, especially in LMICs such as South Africa.

Corrigendum: Stuttering in individuals with Down syndrome: a systematic review of earlier research.

[This corrects the article DOI: 10.3389/fpsyg.2023.1176743.].

Requirements for the Acceptance of an App for Voice Therapy-A Usability Report Based on the "Oldenburger Logopädie App" (OLA).

OBJECTIVE: The prototype "Oldenburger Logopädie App" (OLA) was designed to support voice therapy for patients with recurrent paresis, such as to accompany homework or as a short-term substitute for regular therapy due to dropouts, such as during the COVID-19 pandemic. The treating speech and language pathologists (SLPs) unlocks videos individually applicable to the respective patients, in which the SLPs instruct the individual exercises. The app can be used without information technology knowledge or detailed instructions. MATERIALS AND METHODS: The prototype's usability was evaluated through a usability test battery (AttrakDiff questionnaire, System Usability Scale, Visual Aesthetics of Websites Inventory questionnaire) and informal interviews from the perspective of patients and SLPs. RESULTS: The acceptance, usability, user experience, self-descriptiveness, and user behavior of OLA were consistently given and mostly rated as positive. Both user groups rated OLA as practical and easy to use (eg, System Usability Scale: "practical" (agree: ∅ 49.5%), "cumbersome to use" (total: strongly disagree: ∅ 60.0%). However, the monotonous layout of the app and the instructional and exercise videos should be modified in the next editing step. An overview of relevant criteria for a voice therapy app, regarding design and functions, was derived from the results. CONCLUSION: This user-oriented feedback on the usability of the voice app provides the proof of concept and the basis for the further development of the Artificial intelligence-based innovative follow-up app LAOLA. In the future, it should be possible to support the treatment of all voice disorders with such an app. For the further development of the voice app, the therapeutic content and the effectiveness of the training should also be investigated.

Children and adults produce distinct technology- and human-directed speech.

This study compares how English-speaking adults and children from the United States adapt their speech when talking to a real person and a smart speaker (Amazon Alexa) in a psycholinguistic experiment. Overall, participants produced more effortful speech when talking to a device (longer duration and higher pitch). These differences also varied by age: children produced even higher pitch in device-directed speech, suggesting a stronger expectation to be misunderstood by the system. In support of this, we see that after a staged recognition error by the device, children increased pitch even more. Furthermore, both adults and children displayed the same degree of variation in their responses for whether "Alexa seems like a real person or not", further indicating that children's conceptualization of the system's competence shaped their register adjustments, rather than an increased anthropomorphism response. This work speaks to models on the mechanisms underlying speech production, and human-computer interaction frameworks, providing support for routinized theories of spoken interaction with technology.

The ethical significance of user-control in AI-driven speech-BCIs: a narrative review.

AI-driven brain-computed interfaces aimed at restoring speech for individuals living with locked-in-syndrome are paired with ethical implications for user's autonomy, privacy and responsibility. Embedding options for sufficient levels of user-control in speech-BCI design has been proposed to mitigate these ethical challenges. However, how user-control in speech-BCIs is conceptualized and how it relates to these ethical challenges is underdetermined. In this narrative literature review, we aim to clarify and explicate the notion of user-control in speech-BCIs, to better understand in what way user-control could operationalize user's autonomy, privacy and responsibility and explore how such suggestions for increasing user-control can be translated to recommendations for the design or use of speech-BCIs. First, we identified types of user control, including executory control that can protect voluntariness of speech, and guidance control that can contribute to semantic accuracy. Second, we identified potential causes for a loss of user-control, including contributions of predictive language models, a lack of ability for neural control, or signal interference and external control. Such a loss of user control may have implications for semantic accuracy and mental privacy. Third we explored ways to design for user-control. While embedding initiation signals for users may increase executory control, they may conflict with other aims such as speed and continuity of speech. Design mechanisms for guidance control remain largely conceptual, similar trade-offs in design may be expected. We argue that preceding these trade-offs, the overarching aim of speech-BCIs needs to be defined, requiring input from current and potential users. Additionally, conceptual clarification of user-control and other (ethical) concepts in this debate has practical relevance for BCI researchers. For instance, different concepts of inner speech may have distinct ethical implications. Increased clarity of such concepts can improve anticipation of ethical implications of speech-BCIs and may help to steer design decisions.

Comparison of Speech Defects in Different Types of Malocclusion.

Introduction Speech has a great impact on human evolution, allowing for the widespread knowledge and advancement of tools. Difficulty in pronouncing one or more sounds is the most common speech impairment. Speech defects are more commonly associated with class III malocclusion patients (difficulty in pronouncing 's' and 't' sounds), the second in line is class II malocclusion (difficulty in pronouncing 's' and 'z' sounds), and speech distortions are least affected in class I malocclusion (difficulty in pronouncing 's' and 'Sh'). Most patients with dentofacial disharmonies and speech distortions need orthodontic care and orthognathic surgery to resolve their issues with mastication, aesthetics, and speech. Aims and objectives To compare and assess speech difficulties in different types of malocclusion. Materials and methods The study was conducted over 160 subjects for three and half months. All of them were evaluated for speech defects before they received orthodontic treatment. The main basis of this study is according to Angle's classification of malocclusion. The subjects were segregated according to Angle's classification of malocclusion. Malocclusion traits that are included in this study are Angle's class I, Angle's class II division I and division II, and Angle's class III. Results According to the results, out of 160 subjects, labio-dental speech defects are observed in 8% where n=13 of the study participants, linguodental speech defects are observed in 2% where n=3, lingua-alveolar speech defects are present in 54% where n=86, and bilabial speech defects are observed in 2% where n=3 of the study participants. Here 'n' represents the frequency of the subjects. Severe speech defects are seen in Angle's class III malocclusion. Results according to the type of malocclusion include: labio-dental speech defects are seen in 37.5% in class I, 25% in class II division I, 0% in class II division II, and 37.5% in class III. Linguodental speech defects are seen in class III malocclusion subjects only. Lingua-alveolar sounds are seen in 27.8% of class I, 29.6% of class II division I, 1.9% of class II division II, and 40.7% of class III. Bilabial speech defects are only seen in class II division I subjects. According to the results, only lingua-alveolar speech defects are statistically significant, and more severe speech defects were observed in class III malocclusion. Conclusion Speech plays an important role in affecting the quality of life of people. Different types of malocclusion traits are associated with different types of speech defects.

Speech Emotion Recognition Incorporating Relative Difficulty and Labeling Reliability.

Emotions in speech are expressed in various ways, and the speech emotion recognition (SER) model may perform poorly on unseen corpora that contain different emotional factors from those expressed in training databases. To construct an SER model robust to unseen corpora, regularization approaches or metric losses have been studied. In this paper, we propose an SER method that incorporates relative difficulty and labeling reliability of each training sample. Inspired by the Proxy-Anchor loss, we propose a novel loss function which gives higher gradients to the samples for which the emotion labels are more difficult to estimate among those in the given minibatch. Since the annotators may label the emotion based on the emotional expression which resides in the conversational context or other modality but is not apparent in the given speech utterance, some of the emotional labels may not be reliable and these unreliable labels may affect the proposed loss function more severely. In this regard, we propose to apply label smoothing for the samples misclassified by a pre-trained SER model. Experimental results showed that the performance of the SER on unseen corpora was improved by adopting the proposed loss function with label smoothing on the misclassified data.

Wireless Mouth Motion Recognition System Based on EEG-EMG Sensors for Severe Speech Impairments.

This study aims to demonstrate the feasibility of using a new wireless electroencephalography (EEG)-electromyography (EMG) wearable approach to generate characteristic EEG-EMG mixed patterns with mouth movements in order to detect distinct movement patterns for severe speech impairments. This paper describes a method for detecting mouth movement based on a new signal processing technology suitable for sensor integration and machine learning applications. This paper examines the relationship between the mouth motion and the brainwave in an effort to develop nonverbal interfacing for people who have lost the ability to communicate, such as people with paralysis. A set of experiments were conducted to assess the efficacy of the proposed method for feature selection. It was determined that the classification of mouth movements was meaningful. EEG-EMG signals were also collected during silent mouthing of phonemes. A few-shot neural network was trained to classify the phonemes from the EEG-EMG signals, yielding classification accuracy of 95%. This technique in data collection and processing bioelectrical signals for phoneme recognition proves a promising avenue for future communication aids.

Enhancing Monitoring Performance: A Microservices Approach to Monitoring with Spyware Techniques and Prediction Models.

In today's digital landscape, organizations face significant challenges, including sensitive data leaks and the proliferation of hate speech, both of which can lead to severe consequences such as financial losses, reputational damage, and psychological impacts on employees. This work considers a comprehensive solution using a microservices architecture to monitor computer usage within organizations effectively. The approach incorporates spyware techniques to capture data from employee computers and a web application for alert management. The system detects data leaks, suspicious behaviors, and hate speech through efficient data capture and predictive modeling. Therefore, this paper presents a comparative performance analysis between Spring Boot and Quarkus, focusing on objective metrics and quantitative statistics. By utilizing recognized tools and benchmarks in the computer science community, the study provides an in-depth understanding of the performance differences between these two platforms. The implementation of Quarkus over Spring Boot demonstrated substantial improvements: memory usage was reduced by up to 80% and CPU usage by 95%, and system uptime decreased by 119%. This solution offers a robust framework for enhancing organizational security and mitigating potential threats through proactive monitoring and predictive analysis while also guiding developers and software architects in making informed technological choices.

Implementation of Sound Direction Detection and Mixed Source Separation in Embedded Systems.

In recent years, embedded system technologies and products for sensor networks and wearable devices used for monitoring people's activities and health have become the focus of the global IT industry. In order to enhance the speech recognition capabilities of wearable devices, this article discusses the implementation of audio positioning and enhancement in embedded systems using embedded algorithms for direction detection and mixed source separation. The two algorithms are implemented using different embedded systems: direction detection developed using TI TMS320C6713 DSK and mixed source separation developed using Raspberry Pi 2. For mixed source separation, in the first experiment, the average signal-to-interference ratio (SIR) at 1 m and 2 m distances was 16.72 and 15.76, respectively. In the second experiment, when evaluated using speech recognition, the algorithm improved speech recognition accuracy to 95%.

Decoding loneliness: Can explainable AI help in understanding language differences in lonely older adults?

STUDY OBJECTIVES: Loneliness impacts the health of many older adults, yet effective and targeted interventions are lacking. Compared to surveys, speech data can capture the personalized experience of loneliness. In this proof-of-concept study, we used Natural Language Processing to extract novel linguistic features and AI approaches to identify linguistic features that distinguish lonely adults from non-lonely adults. METHODS: Participants completed UCLA loneliness scales and semi-structured interviews (sections: social relationships, loneliness, successful aging, meaning/purpose in life, wisdom, technology and successful aging). We used the Linguistic Inquiry and Word Count (LIWC-22) program to analyze linguistic features and built a classifier to predict loneliness. Each interview section was analyzed using an explainable AI (XAI) model to classify loneliness. RESULTS: The sample included 97 older adults (age 66-101 years, 65 % women). The model had high accuracy (Accuracy: 0.889, AUC: 0.8), precision (F1: 0.8), and recall (1.0). The sections on social relationships and loneliness were most important for classifying loneliness. Social themes, conversational fillers, and pronoun usage were important features for classifying loneliness. CONCLUSIONS: XAI approaches can be used to detect loneliness through the analyses of unstructured speech and to better understand the experience of loneliness.

The contribution of short-term memory for sound features to speech-in-noise perception and cognition.

Speech-in-noise (SIN) perception is a fundamental ability that declines with aging, as does general cognition. We assess whether auditory cognitive ability, in particular short-term memory for sound features, contributes to both. We examined how auditory memory for fundamental sound features, the carrier frequency and amplitude modulation rate of modulated white noise, contributes to SIN perception. We assessed SIN in 153 healthy participants with varying degrees of hearing loss using measures that require single-digit perception (the Digits-in-Noise, DIN) and sentence perception (Speech-in-Babble, SIB). Independent variables were auditory memory and a range of other factors including the Pure Tone Audiogram (PTA), a measure of dichotic pitch-in-noise perception (Huggins pitch), and demographic variables including age and sex. Multiple linear regression models were compared using Bayesian Model Comparison. The best predictor model for DIN included PTA and Huggins pitch (r2 = 0.32, p < 0.001), whereas the model for SIB included the addition of auditory memory for sound features (r2 = 0.24, p < 0.001). Further analysis demonstrated that auditory memory also explained a significant portion of the variance (28 %) in scores for a screening cognitive test for dementia. Auditory memory for non-speech sounds may therefore provide an important predictor of both SIN and cognitive ability.

Exploring the conceptualisation, measurement, clinical utility and treatment of formal thought disorder in psychosis: A Delphi study.

BACKGROUND: Formal Thought Disorder (FTD) is a recognised psychiatric symptom, yet its characterisation remains debated. This is problematic because it contributes to poor efficiency and heterogeneity in psychiatric research, with salient clinical impact. OBJECTIVE: This study aimed to investigate expert opinion on the concept, measurement and clinical utility of FTD using the Delphi technique. METHOD: Across three rounds, experts were queried on their definitions of FTD, methods for the assessment and measurement of FTD, associated clinical outcomes and treatment options. RESULTS: Responses were obtained from 56 experts, demonstrating varying levels of consensus across different aspects of FTD. While consensus (>80 %) was reached for some aspects on the concept of FTD, including its definition and associated symptomology and mechanisms, others remained less clear. Overall, the universal importance attributed to the clinical understanding, measurement and treatment of FTD was clear, although consensus was infrequent as to the reasons behind and methods for doing so. CONCLUSIONS: Our results contribute to the still elusive formal definition of FTD. The multitude of interpretations regarding these topics highlights the need for further clarity with this phenomenon. Our findings emphasised that the measurement and clinical utility of FTD are closely tied to the concept; hence, until there is agreement on the concept of FTD, difficulties with measuring and understanding its clinical usefulness to inform treatment interventions will persist. Future FTD research should focus on clarifying the factor structure and dimensionality to determine the latent structure and elucidate the core clinical phenotype.

Cross-corpus speech emotion recognition with transformers: Leveraging handcrafted features and data augmentation.

Speech emotion recognition (SER) stands as a prominent and dynamic research field in data science due to its extensive application in various domains such as psychological assessment, mobile services, and computer games, mobile services. In previous research, numerous studies utilized manually engineered features for emotion classification, resulting in commendable accuracy. However, these features tend to underperform in complex scenarios, leading to reduced classification accuracy. These scenarios include: 1. Datasets that contain diverse speech patterns, dialects, accents, or variations in emotional expressions. 2. Data with background noise. 3. Scenarios where the distribution of emotions varies significantly across datasets can be challenging. 4. Combining datasets from different sources introduce complexities due to variations in recording conditions, data quality, and emotional expressions. Consequently, there is a need to improve the classification performance of SER techniques. To address this, a novel SER framework was introduced in this study. Prior to feature extraction, signal preprocessing and data augmentation methods were applied to augment the available data, resulting in the derivation of 18 informative features from each signal. The discriminative feature set was obtained using feature selection techniques which was then utilized as input for emotion recognition using the SAVEE, RAVDESS, and EMO-DB datasets. Furthermore, this research also implemented a cross-corpus model that incorporated all speech files related to common emotions from three datasets. The experimental outcomes demonstrated the superior performance of SER framework compared to existing frameworks in the field. Notably, the framework presented in this study achieved remarkable accuracy rates across various datasets. Specifically, the proposed model obtained an accuracy of 95%, 94%,97%, and 97% on SAVEE, RAVDESS, EMO-DB and cross-corpus datasets respectively. These results underscore the significant contribution of our proposed framework to the field of SER.

The developmental pattern of native and non-native speech perception during the 1st year of life in Japanese infants.

Language development during the 1st year of life is characterized by perceptual attunement: following language-general perception, a decline in the perception of non-native phonemes and a parallel increase in or maintenance of the perception of native phonemes. While this general pattern is well established, there are still many gaps in the literature. First, most evidence documenting these patterns comes from "Minority world countries" with only a limited number of studies from "Majority world countries", limiting the range of languages and contrasts assessed. Second, few studies test both the developmental patterns of native and non-native speech perception in the same group of infants, making it hard to draw conclusions on simultaneous decline in non-native and increase in native speech perception. Such limitations are in part due to the effort that goes into testing developing speech sound perception, where usually only discrimination of one contrast per infant can be tested at a time. The present study thus set out to assess the feasibility of assessing a given infant on their discrimination of two speech sound contrasts during the same lab visit. It leveraged the presence of documented patterns of the improvement of native and the decline of non-native phoneme discrimination abilities in Japanese, therefore assessing native and non-native speech perception in Japanese infants from 6 to 12 months of age. Results demonstrated that 76 % of infants contributed discrimination data for both contrasts. We found a decline in non-native speech perception evident in discrimination of the non-native /ɹ/-/l/ consonant contrast at 9-11, but not at 11-13 months of age. Additionally, a parallel increase in native speech perception was demonstrated evident in an absence of native phonemic vowel length discrimination at 6-7 and 9-11 months and a discrimination of this contrast at 11-13 months of age. These results, based on a simultaneous assessment of native and non-native speech perception in Japanese-learning infants, demonstrate the feasibility of assessing the discrimination of two contrasts in one testing session and corroborate theoretical proposals on two hallmarks of perceptual attunement: a decrease in non-native and a facilitation in native speech perception during the first year of life.

Multimodal Speech Biomarkers for Remote Monitoring of ALS Disease Progression.

Amyotrophic lateral sclerosis (ALS) is a progressive neurodegenerative disease that severely impacts affected persons' speech and motor functions, yet early detection and tracking of disease progression remain challenging. The current gold standard for monitoring ALS progression, the ALS functional rating scale - revised (ALSFRS-R), is based on subjective ratings of symptom severity, and may not capture subtle but clinically meaningful changes due to a lack of granularity. Multimodal speech measures which can be automatically collected from patients in a remote fashion allow us to bridge this gap because they are continuous-valued and therefore, potentially more granular at capturing disease progression. Here we investigate the responsiveness and sensitivity of multimodal speech measures in persons with ALS (pALS) collected via a remote patient monitoring platform in an effort to quantify how long it takes to detect a clinically-meaningful change associated with disease progression. We recorded audio and video from 278 participants and automatically extracted multimodal speech biomarkers (acoustic, orofacial, linguistic) from the data. We find that the timing alignment of pALS speech relative to a canonical elicitation of the same prompt and the number of words used to describe a picture are the most responsive measures at detecting such change in both pALS with bulbar (n = 36) and non-bulbar onset (n = 107). Interestingly, the responsiveness of these measures is stable even at small sample sizes. We further found that certain speech measures are sensitive enough to track bulbar decline even when there is no patient-reported clinical change, i.e. the ALSFRS-R speech score remains unchanged at 3 out of a total possible score of 4. The findings of this study have the potential to facilitate improved, accelerated and cost-effective clinical trials and care.

Evaluating the accuracy of automated processing of child and adult language production in preschool classrooms.

Young children's language and social development is influenced by the linguistic environment of their classrooms, including their interactions with teachers and peers. Measurement of the classroom linguistic environment typically relies on observational methods, often providing limited 'snapshots' of children's interactions, from which broad generalizations are made. Recent technological advances, including artificial intelligence, provide opportunities to capture children's interactions using continuous recordings representing much longer durations of time. The goal of the present study was to evaluate the accuracy of the Interaction Detection in Early Childhood Settings (IDEAS) system on 13 automated indices of language output using recordings collected from 19 children and three teachers over two weeks in an urban preschool classroom. The accuracy of language outputs processed via IDEAS were compared to ground truth via linear correlations and median absolute relative error. Findings indicate high correlations between IDEAS and ground truth data on measures of teacher and child speech, and relatively low error rates on the majority of IDEAS language output measures. Study findings indicate that IDEAS may provide a useful measurement tool for advancing knowledge about children's classroom experiences and their role in shaping development.