HEAR4Health: a blueprint for making computer audition a staple of modern healthcare.

Recent years have seen a rapid increase in digital medicine research in an attempt to transform traditional healthcare systems to their modern, intelligent, and versatile equivalents that are adequately equipped to tackle contemporary challenges. This has led to a wave of applications that utilise AI technologies; first and foremost in the fields of medical imaging, but also in the use of wearables and other intelligent sensors. In comparison, computer audition can be seen to be lagging behind, at least in terms of commercial interest. Yet, audition has long been a staple assistant for medical practitioners, with the stethoscope being the quintessential sign of doctors around the world. Transforming this traditional technology with the use of AI entails a set of unique challenges. We categorise the advances needed in four key pillars: Hear, corresponding to the cornerstone technologies needed to analyse auditory signals in real-life conditions; Earlier, for the advances needed in computational and data efficiency; Attentively, for accounting to individual differences and handling the longitudinal nature of medical data; and, finally, Responsibly, for ensuring compliance to the ethical standards accorded to the field of medicine. Thus, we provide an overview and perspective of HEAR4Health: the sketch of a modern, ubiquitous sensing system that can bring computer audition on par with other AI technologies in the strive for improved healthcare systems.

Vocalisation Repertoire at the End of the First Year of Life: An Exploratory Comparison of Rett Syndrome and Typical Development.

Rett syndrome (RTT) is a rare, late detected developmental disorder associated with severe deficits in the speech-language domain. Despite a few reports about atypicalities in the speech-language development of infants and toddlers with RTT, a detailed analysis of the pre-linguistic vocalisation repertoire of infants with RTT is yet missing. Based on home video recordings, we analysed the vocalisations between 9 and 11 months of age of three female infants with typical RTT and compared them to three age-matched typically developing (TD) female controls. The video material of the infants had a total duration of 424 min with 1655 infant vocalisations. For each month, we (1) calculated the infants' canonical babbling ratios with CBRUTTER, i.e., the ratio of number of utterances containing canonical syllables to total number of utterances, and (2) classified their pre-linguistic vocalisations in three non-canonical and four canonical vocalisation subtypes. All infants achieved the milestone of canonical babbling at 9 months of age according to their canonical babbling ratios, i.e. CBRUTTER ≥ 0.15. We revealed overall lower CBRsUTTER and a lower proportion of canonical pre-linguistic vocalisations consisting of well-formed sounds that could serve as parts of target-language words for the RTT group compared to the TD group. Further studies with more data from individuals with RTT are needed to study the atypicalities in the pre-linguistic vocalisation repertoire which may portend the later deficits in spoken language that are characteristic features of RTT.

Triplet Loss-Based Models for COVID-19 Detection from Vocal Sounds.

This work focuses on the automatic detection of COVID-19 from the analysis of vocal sounds, including sustained vowels, coughs, and speech while reading a short text. Specifically, we use the Mel-spectrogram representations of these acoustic signals to train neural network-based models for the task at hand. The extraction of deep learnt representations from the Mel-spectrograms is performed with Convolutional Neural Networks (CNNs). In an attempt to guide the training of the embedded representations towards more separable and robust inter-class representations, we explore the use of a triplet loss function. The experiments performed are conducted using the Your Voice Counts dataset, a new dataset containing German speakers collected using smartphones. The results obtained support the suitability of using triplet loss-based models to detect COVID-19 from vocal sounds. The best Unweighted Average Recall (UAR) of 66.5 % is obtained using a triplet loss-based model exploiting vocal sounds recorded while reading.

Automatic vocalisation-based detection of fragile X syndrome and Rett syndrome.

Fragile X syndrome (FXS) and Rett syndrome (RTT) are developmental disorders currently not diagnosed before toddlerhood. Even though speech-language deficits are among the key symptoms of both conditions, little is known about infant vocalisation acoustics for an automatic earlier identification of affected individuals. To bridge this gap, we applied intelligent audio analysis methodology to a compact dataset of 4454 home-recorded vocalisations of 3 individuals with FXS and 3 individuals with RTT aged 6 to 11 months, as well as 6 age- and gender-matched typically developing controls (TD). On the basis of a standardised set of 88 acoustic features, we trained linear kernel support vector machines to evaluate the feasibility of automatic classification of (a) FXS vs TD, (b) RTT vs TD, (c) atypical development (FXS+RTT) vs TD, and (d) FXS vs RTT vs TD. In paradigms (a)-(c), all infants were correctly classified; in paradigm (d), 9 of 12 were so. Spectral/cepstral and energy-related features were most relevant for classification across all paradigms. Despite the small sample size, this study reveals new insights into early vocalisation characteristics in FXS and RTT, and provides technical underpinnings for a future earlier identification of affected individuals, enabling earlier intervention and family counselling.

The Acoustic Dissection of Cough: Diving Into Machine Listening-based COVID-19 Analysis and Detection.

OBJECTIVES: The coronavirus disease 2019 (COVID-19) has caused a crisis worldwide. Amounts of efforts have been made to prevent and control COVID-19's transmission, from early screenings to vaccinations and treatments. Recently, due to the spring up of many automatic disease recognition applications based on machine listening techniques, it would be fast and cheap to detect COVID-19 from recordings of cough, a key symptom of COVID-19. To date, knowledge of the acoustic characteristics of COVID-19 cough sounds is limited but would be essential for structuring effective and robust machine learning models. The present study aims to explore acoustic features for distinguishing COVID-19 positive individuals from COVID-19 negative ones based on their cough sounds. METHODS: By applying conventional inferential statistics, we analyze the acoustic correlates of COVID-19 cough sounds based on the ComParE feature set, i.e., a standardized set of 6,373 acoustic higher-level features. Furthermore, we train automatic COVID-19 detection models with machine learning methods and explore the latent features by evaluating the contribution of all features to the COVID-19 status predictions. RESULTS: The experimental results demonstrate that a set of acoustic parameters of cough sounds, e.g., statistical functionals of the root mean square energy and Mel-frequency cepstral coefficients, bear essential acoustic information in terms of effect sizes for the differentiation between COVID-19 positive and COVID-19 negative cough samples. Our general automatic COVID-19 detection model performs significantly above chance level, i.e., at an unweighted average recall (UAR) of 0.632, on a data set consisting of 1,411 cough samples (COVID-19 positive/negative: 210/1,201). CONCLUSIONS: Based on the acoustic correlates analysis on the ComParE feature set and the feature analysis in the effective COVID-19 detection approach, we find that several acoustic features that show higher effects in conventional group difference testing are also higher weighted in the machine learning models.

Is Speech the New Blood? Recent Progress in AI-Based Disease Detection From Audio in a Nutshell.

In recent years, advancements in the field of artificial intelligence (AI) have impacted several areas of research and application. Besides more prominent examples like self-driving cars or media consumption algorithms, AI-based systems have further started to gain more and more popularity in the health care sector, however whilst being restrained by high requirements for accuracy, robustness, and explainability. Health-oriented AI research as a sub-field of digital health investigates a plethora of human-centered modalities. In this article, we address recent advances in the so far understudied but highly promising audio domain with a particular focus on speech data and present corresponding state-of-the-art technologies. Moreover, we give an excerpt of recent studies on the automatic audio-based detection of diseases ranging from acute and chronic respiratory diseases via psychiatric disorders to developmental disorders and neurodegenerative disorders. Our selection of presented literature shows that the recent success of deep learning methods in other fields of AI also more and more translates to the field of digital health, albeit expert-designed feature extractors and classical ML methodologies are still prominently used. Limiting factors, especially for speech-based disease detection systems, are related to the amount and diversity of available data, e. g., the number of patients and healthy controls as well as the underlying distribution of age, languages, and cultures. Finally, we contextualize and outline application scenarios of speech-based disease detection systems as supportive tools for health-care professionals under ethical consideration of privacy protection and faulty prediction.

The voice of COVID-19: Acoustic correlates of infection in sustained vowels.

COVID-19 is a global health crisis that has been affecting our daily lives throughout the past year. The symptomatology of COVID-19 is heterogeneous with a severity continuum. Many symptoms are related to pathological changes in the vocal system, leading to the assumption that COVID-19 may also affect voice production. For the first time, the present study investigates voice acoustic correlates of a COVID-19 infection based on a comprehensive acoustic parameter set. We compare 88 acoustic features extracted from recordings of the vowels /i:/, /e:/, /u:/, /o:/, and /a:/ produced by 11 symptomatic COVID-19 positive and 11 COVID-19 negative German-speaking participants. We employ the Mann-Whitney U test and calculate effect sizes to identify features with prominent group differences. The mean voiced segment length and the number of voiced segments per second yield the most important differences across all vowels indicating discontinuities in the pulmonic airstream during phonation in COVID-19 positive participants. Group differences in front vowels are additionally reflected in fundamental frequency variation and the harmonics-to-noise ratio, group differences in back vowels in statistics of the Mel-frequency cepstral coefficients and the spectral slope. Our findings represent an important proof-of-concept contribution for a potential voice-based identification of individuals infected with COVID-19.

Efficient Collection and Representation of Preverbal Data in Typical and Atypical Development.

Human preverbal development refers to the period of steadily increasing vocal capacities until the emergence of a child's first meaningful words. Over the last decades, research has intensively focused on preverbal behavior in typical development. Preverbal vocal patterns have been phonetically classified and acoustically characterized. More recently, specific preverbal phenomena were discussed to play a role as early indicators of atypical development. Recent advancements in audio signal processing and machine learning have allowed for novel approaches in preverbal behavior analysis including automatic vocalization-based differentiation of typically and atypically developing individuals. In this paper, we give a methodological overview of current strategies for collecting and acoustically representing preverbal data for intelligent audio analysis paradigms. Efficiency in the context of data collection and data representation is discussed. Following current research trends, we set a special focus on challenges that arise when dealing with preverbal data of individuals with late detected developmental disorders, such as autism spectrum disorder or Rett syndrome.

Early motor and pre-linguistic verbal development in Prader-Willi syndrome - A case report.

BACKGROUND: Prader-Willi syndrome (PWS) is a rare genetic disorder. Infants with PWS show a neurodevelopmental dysfunction which entails a delayed motor and language development, but studies on their spontaneous movements (i.e. general movements) or pre-linguistic speech-language development before 6 months of age are missing so far. AIM: To describe early motor and pre-linguistic verbal development in an infant with PWS. METHODS AND PROCEDURES: Prospective case report; in addition to the assessment of general movements and the concurrent movement repertoire, we report on early verbal forms, applying the Stark Assessment of Early Vocal Development-Revised. OUTCOMES AND RESULTS: General movements were abnormal on days 8 and 15. No fidgety movements were observed at 11 weeks; they only emerged at 17 weeks and lasted until at least 27 weeks post-term. The movement character was monotonous, and early motor milestones were only achieved with a delay. At 27 weeks the infant produced age-adequate types of vocalisations. However, none of the canonical-syllable vocalisations that typically emerge at that age were observed. Early vocalisations appeared monotonous and with a peculiarly harmonic structure. CONCLUSIONS AND IMPLICATIONS: Early motor and pre-linguistic verbal behaviours were monotonous in an infant with PWS throughout his first 6 months of life. This suggests that early signs of neurodevelopmental dysfunction (i.e. abnormal general movements) might already be diagnosed in infants with PWS during their first weeks of life, potentially enabling us to diagnose and intervene at an early stage.

Canonical Babbling: A Marker for Earlier Identification of Late Detected Developmental Disorders?

PURPOSE OF REVIEW: To summarize findings about the emergence and characteristics of canonical babbling in children with late detected developmental disorders (LDDDs), such as autism spectrum disorder, Rett syndrome, and fragile X syndrome. In particular, we ask whether infants' vocal development in the first year of life contains any markers that may contribute to earlier detection of these disorders. RECENT FINDINGS: Only a handful studies have investigated canonical babbling in infants with LDDDs. With divergent research paradigms and definitions applied, findings on the onset and characteristics of canonical babbling are inconsistent and difficult to compare. Infants with LDDDs showed reduced likelihood to produce canonical babbling vocalizations. If achieved, this milestone was more likely to be reached beyond the critical time window of 5-10 months. SUMMARY: Canonical babbling appears promising as a potential marker for early detection of infants at risk for developmental disorders. In-depth studies on babbling characteristics in LDDDs are warranted.

Early Vocal Development in Autism Spectrum Disorder, Rett Syndrome, and Fragile X Syndrome: Insights from Studies using Retrospective Video Analysis.

This article provides an overview of studies assessing the early vocalisations of children with autism spectrum disorder (ASD), Rett syndrome (RTT), and fragile X syndrome (FXS) using retrospective video analysis (RVA) during the first two years of life. Electronic databases were systematically searched and a total of 23 studies were selected. These studies were then categorised according to whether children were later diagnosed with ASD (13 studies), RTT (8 studies), or FXS (2 studies), and then described in terms of (a) participant characteristics, (b) control group characteristics, (c) video footage, (d) behaviours analysed, and (e) main findings. This overview supports the use of RVA in analysing the early development of vocalisations in children later diagnosed with ASD, RTT or FXS, and provides an in-depth analysis of vocalisation presentation, complex vocalisation production, and the rate and/or frequency of vocalisation production across the three disorders. Implications are discussed in terms of extending crude vocal analyses to more precise methods that might provide more powerful means by which to discriminate between disorders during early development. A greater understanding of the early manifestation of these disorders may then lead to improvements in earlier detection.

Response to name and its value for the early detection of developmental disorders: Insights from autism spectrum disorder, Rett syndrome, and fragile X syndrome. A perspectives paper.

BACKGROUND: Responding to one's own name (RtN) has been reported as atypical in children with developmental disorders, yet comparative studies on RtN across syndromes are rare. AIMS: We aim to (a) overview the literature on RtN in different developmental disorders during the first 24 months of life, and (b) report comparative data on RtN across syndromes. METHODS AND PROCEDURES: In Part 1, a literature search, focusing on RtN in children during the first 24 months of life with developmental disorders, identified 23 relevant studies. In Part 2, RtN was assessed utilizing retrospective video analysis for infants later diagnosed with ASD, RTT, or FXS, and typically developing peers. OUTCOMES AND RESULTS: Given a variety of methodologies and instruments applied to assess RtN, 21/23 studies identified RtN as atypical in infants with a developmental disorder. We observed four different developmental trajectories of RtN in ASD, RTT, PSV, and FXS from 9 to 24 months of age. Between-group differences became more distinctive with age. CONCLUSIONS AND IMPLICATIONS: RtN may be a potential parameter of interest in a comprehensive early detection model characterising age-specific neurofunctional biomarkers associated with specific disorders, and contribute to early identification.

Typical vs. atypical: Combining auditory Gestalt perception and acoustic analysis of early vocalisations in Rett syndrome.

BACKGROUND: Early speech-language development of individuals with Rett syndrome (RTT) has been repeatedly characterised by a co-occurrence of apparently typical and atypical vocalisations. AIMS: To describe specific features of this intermittent character of typical versus atypical early RTT-associated vocalisations by combining auditory Gestalt perception and acoustic vocalisation analysis. METHODS AND PROCEDURES: We extracted N = 363 (pre-)linguistic vocalisations from home video recordings of an infant later diagnosed with RTT. In a listening experiment, all vocalisations were assessed for (a)typicality by five experts on early human development. Listeners' auditory concepts of (a)typicality were investigated in context of a comprehensive set of acoustic time-, spectral- and/or energy-related higher-order features extracted from the vocalisations. OUTCOMES AND RESULTS: More than half of the vocalisations were rated as 'atypical' by at least one listener. Atypicality was mainly related to the auditory attribute 'timbre', and to prosodic, spectral, and voice quality features in the acoustic domain. CONCLUSIONS AND IMPLICATIONS: Knowledge gained in our study shall contribute to the generation of an objective model of early vocalisation atypicality. Such a model might be used for increasing caregivers' and healthcare professionals' sensitivity to identify atypical vocalisation patterns, or even for a probabilistic approach to automatically detect RTT based on early vocalisations.

A Novel Way to Measure and Predict Development: A Heuristic Approach to Facilitate the Early Detection of Neurodevelopmental Disorders.

PURPOSE OF REVIEW: Substantial research exists focusing on the various aspects and domains of early human development. However, there is a clear blind spot in early postnatal development when dealing with neurodevelopmental disorders, especially those that manifest themselves clinically only in late infancy or even in childhood. RECENT FINDINGS: This early developmental period may represent an important timeframe to study these disorders but has historically received far less research attention. We believe that only a comprehensive interdisciplinary approach will enable us to detect and delineate specific parameters for specific neurodevelopmental disorders at a very early age to improve early detection/diagnosis, enable prospective studies and eventually facilitate randomised trials of early intervention. In this article, we propose a dynamic framework for characterising neurofunctional biomarkers associated with specific disorders in the development of infants and children. We have named this automated detection 'Fingerprint Model', suggesting one possible approach to accurately and early identify neurodevelopmental disorders.

Eye Movements during Silent and Oral Reading in a Regular Orthography: Basic Characteristics and Correlations with Childhood Cognitive Abilities and Adolescent Reading Skills.

The present study aimed to define differences between silent and oral reading with respect to spatial and temporal eye movement parameters. Eye movements of 22 German-speaking adolescents (14 females; mean age = 13;6 years;months) were recorded while reading an age-appropriate text silently and orally. Preschool cognitive abilities were assessed at the participants' age of 5;7 (years;months) using the Kaufman Assessment Battery for Children. The participants' reading speed and reading comprehension at the age of 13;6 (years;months) were determined using a standardized inventory to evaluate silent reading skills in German readers (Lesegeschwindigkeits- und -verständnistest für Klassen 6-12). The results show that (i) reading mode significantly influenced both spatial and temporal characteristics of eye movement patterns; (ii) articulation decreased the consistency of intraindividual reading performances with regard to a significant number of eye movement parameters; (iii) reading skills predicted the majority of eye movement parameters during silent reading, but influenced only a restricted number of eye movement parameters when reading orally; (iv) differences with respect to a subset of eye movement parameters increased with reading skills; (v) an overall preschool cognitive performance score predicted reading skills at the age of 13;6 (years;months), but not eye movement patterns during either silent or oral reading. However, we found a few significant correlations between preschool performances on subscales of sequential and simultaneous processing and eye movement parameters for both reading modes. Overall, the findings suggest that eye movement patterns depend on the reading mode. Preschool cognitive abilities were more closely related to eye movement patterns of oral than silent reading, while reading skills predicted eye movement patterns during silent reading, but less so during oral reading.

Parents' initial concerns about the development of their children later diagnosed with fragile X syndrome.

BACKGROUND: Retrospective parental reports have often been used to identify the early characteristics of children later diagnosed with a developmental disorder. METHOD: We applied this methodology to document 13 parents' initial concerns about the development of their 17 children later diagnosed with fragile X syndrome (FXS). Parents were additionally asked about when they noticed the emergence of behavioural signs related to FXS. RESULTS: More than half of the parents reported initial concerns prior to the child's first birthday and in most cases it was deviant motor behaviours that caused the first concerns. Behavioural signs related to the FXS phenotype were also reported to be perceptible in the first year of the child's life. CONCLUSIONS: Due to limitations of retrospective parental questionnaires, we suggest that other methodologies, such as home video analysis, are needed to complement our understanding of the pathways of developmental disorders with late clinical onsets.

The Relation between Reading Skills and Eye Movement Patterns in Adolescent Readers: Evidence from a Regular Orthography.

Over the past decades, the relation between reading skills and eye movement behavior has been well documented in English-speaking cohorts. As English and German differ substantially with regard to orthographic complexity (i.e. grapheme-phoneme correspondence), we aimed to delineate specific characteristics of how reading speed and reading comprehension interact with eye movements in typically developing German-speaking (Austrian) adolescents. Eye movements of 22 participants (14 females; mean age = 13;6 years;months) were tracked while they were performing three tasks, namely silently reading words, texts, and pseudowords. Their reading skills were determined by means of a standardized German reading speed and reading comprehension assessment (Lesegeschwindigkeits- und -verständnistest für Klassen 6-12). We found that (a) reading skills were associated with various eye movement parameters in each of the three reading tasks; (b) better reading skills were associated with an increased efficiency of eye movements, but were primarily linked to spatial reading parameters, such as the number of fixations per word, the total number of saccades and saccadic amplitudes; (c) reading speed was a more reliable predictor for eye movement parameters than reading comprehension; (d) eye movements were highly correlated across reading tasks, which indicates consistent reading performances. Contrary to findings in English-speaking cohorts, the reading skills neither consistently correlated with temporal eye movement parameters nor with the number or percentage of regressions made while performing any of the three reading tasks. These results indicate that, although reading skills are associated with eye movement patterns irrespective of language, the temporal and spatial characteristics of this association may vary with orthographic consistency.

What do home videos tell us about early motor and socio-communicative behaviours in children with autistic features during the second year of life--An exploratory study.

BACKGROUND: Little is known about the first half year of life of individuals later diagnosed with autism spectrum disorders (ASD). There is even a complete lack of observations on the first 6 months of life of individuals with transient autistic behaviours who improved in their socio-communicative functions in the pre-school age. AIM: To compare early development of individuals with transient autistic behaviours and those later diagnosed with ASD. STUDY DESIGN: Exploratory study; retrospective home video analysis. SUBJECTS: 18 males, videoed between birth and the age of 6 months (ten individuals later diagnosed with ASD; eight individuals who lost their autistic behaviours after the age of 3 and achieved age-adequate communicative abilities, albeit often accompanied by tics and attention deficit). METHOD: The detailed video analysis focused on general movements (GMs), the concurrent motor repertoire, eye contact, responsive smiling, and pre-speech vocalisations. RESULTS: Abnormal GMs were observed more frequently in infants later diagnosed with ASD, whereas all but one infant with transient autistic behaviours had normal GMs (p<0.05). Eye contact and responsive smiling were inconspicuous for all individuals. Cooing was not observable in six individuals across both groups. CONCLUSIONS: GMs might be one of the markers which could assist the earlier identification of ASD. We recommend implementing the GM assessment in prospective studies on ASD.

Comparing social reciprocity in preserved speech variant and typical Rett syndrome during the early years of life.

This study compared early markers of social reciprocity in children with typical Rett syndrome (RTT) and in those with the preserved speech variant (PSV) of RTT. Retrospective video analysis of 10 toddlers with typical RTT and five with PSV investigated participants' orientation to their name being called between the ages of 5 and 24 months, prior to their diagnosis. From analysis of the recordings two distinct profiles were apparent. Although response rate was higher in girls with typical RTT than PSV at 5 to 8 months this noticeably reversed from 9 to 12 months onwards. By two years of age there was a markedly higher rate and range of responses from girls with PSV. This study contributes to the delineation of different profiles for the variants of RTT.

Are sporadic fidgety movements as clinically relevant as is their absence?

BACKGROUND: Infants with normal fidgety movements at 3 to 5 months after term are very likely to show neurologically normal development, while the absence of fidgety movements is an early marker for an adverse neurological outcome, mainly cerebral palsy (CP). The clinical significance of so-called sporadic fidgety movements (i.e., fidgety movements occur isolated in a few body parts and are of 1- to 3-second-duration) is not yet known. AIMS: Our objective was to determine whether infants who had developed CP and had sporadic fidgety movements have a better outcome than infants who did not have fidgety movements. STUDY DESIGN: Longitudinal study. Retrospective analysis of prospectively collected data. SUBJECTS: 61 infants who developed CP (46 male, 15 female; 29 infants born preterm; videoed for the assessment of movements and postures at 9 to 16 weeks post-term age). OUTCOME MEASURES: The Gross Motor Function Classification System (GMFCS) was applied at 3 to 5 years of age. RESULTS: There was no difference between children diagnosed with CP who had sporadic fidgety movements at 9 to 16 weeks post-term age (n = 9) and those who never developed fidgety movements (n = 50) with regard to their functional mobility and activity limitation at 3 to 5 years of age. One infant had normal FMs and developed unilateral CP, GMFCS Level I; the remaining infant had abnormal FMs and developed bilateral CP, GMFCS Level II. CONCLUSIONS: There is no evidence that the occurrence of occasional isolated fidgety bursts indicates a milder type of CP.