Extensive data engineering to the rescue: building a multi-species katydid detector from unbalanced, atypical training datasets.

Passive acoustic monitoring (PAM) is a powerful tool for studying ecosystems. However, its effective application in tropical environments, particularly for insects, poses distinct challenges. Neotropical katydids produce complex species-specific calls, spanning mere milliseconds to seconds and spread across broad audible and ultrasonic frequencies. However, subtle differences in inter-pulse intervals or central frequencies are often the only discriminatory traits. These extremities, coupled with low source levels and susceptibility to masking by ambient noise, challenge species identification in PAM recordings. This study aimed to develop a deep learning-based solution to automate the recognition of 31 katydid species of interest in a biodiverse Panamanian forest with over 80 katydid species. Besides the innate challenges, our efforts were also encumbered by a limited and imbalanced initial training dataset comprising domain-mismatched recordings. To overcome these, we applied rigorous data engineering, improving input variance through controlled playback re-recordings and by employing physics-based data augmentation techniques, and tuning signal-processing, model and training parameters to produce a custom well-fit solution. Methods developed here are incorporated into Koogu, an open-source Python-based toolbox for developing deep learning-based bioacoustic analysis solutions. The parametric implementations offer a valuable resource, enhancing the capabilities of PAM for studying insects in tropical ecosystems. This article is part of the theme issue 'Towards a toolkit for global insect biodiversity monitoring'.

Global birdsong embeddings enable superior transfer learning for bioacoustic classification.

Automated bioacoustic analysis aids understanding and protection of both marine and terrestrial animals and their habitats across extensive spatiotemporal scales, and typically involves analyzing vast collections of acoustic data. With the advent of deep learning models, classification of important signals from these datasets has markedly improved. These models power critical data analyses for research and decision-making in biodiversity monitoring, animal behaviour studies, and natural resource management. However, deep learning models are often data-hungry and require a significant amount of labeled training data to perform well. While sufficient training data is available for certain taxonomic groups (e.g., common bird species), many classes (such as rare and endangered species, many non-bird taxa, and call-type) lack enough data to train a robust model from scratch. This study investigates the utility of feature embeddings extracted from audio classification models to identify bioacoustic classes other than the ones these models were originally trained on. We evaluate models on diverse datasets, including different bird calls and dialect types, bat calls, marine mammals calls, and amphibians calls. The embeddings extracted from the models trained on bird vocalization data consistently allowed higher quality classification than the embeddings trained on general audio datasets. The results of this study indicate that high-quality feature embeddings from large-scale acoustic bird classifiers can be harnessed for few-shot transfer learning, enabling the learning of new classes from a limited quantity of training data. Our findings reveal the potential for efficient analyses of novel bioacoustic tasks, even in scenarios where available training data is limited to a few samples.

Quantification by droplet digital PCR and species identification by metabarcoding of environmental (e)DNA from Blainville's beaked whales, with assisted localization from an acoustic array.

Detection and identification of species, subspecies or stocks of whales, dolphins and porpoises at sea remain challenging, particularly for cryptic or elusive species like beaked whales (Family: Ziphiidae). Here we investigated the potential for using an acoustically assisted sampling design to collect environmental (e)DNA from beaked whales on the U.S. Navy's Atlantic Undersea Test and Evaluation Center (AUTEC) in The Bahamas. During 12 days of August 2019, we conducted 9 small-boat surveys and collected 56 samples of seawater (paired subsamples of 1L each, including controls) using both a spatial collection design in the absence of visual confirmation of whales, and a serial collection design in the proximity of whales at the surface. There were 7 sightings of whales, including 11 Blainville's beaked whales (Mesoplodon densirostris). All whales were located initially with the assistance of information from a bottom-mounted acoustic array available on the AUTEC range. Quantification by droplet digital (dd)PCR from the four spatial design collections showed no samples of eDNA above the threshold of detection and none of these 20 samples yielded amplicons for conventional or next-generation sequencing. Quantification of the 31 samples from four serial collections identified 11 likely positive detections. eDNA barcoding by conventional sequencing and eDNA metabarcoding by next-generation sequencing confirmed species identification for 9 samples from three of the four serial collections. We further resolved five intra-specific variants (i.e., haplotypes), two of which showed an exact match to previously published haplotypes and three that have not been reported previously to the international repository, GenBank. A minimum spanning network of the five eDNA haplotypes, with all other published haplotypes of Blainville's beaked whales, suggested the potential for further resolution of differences between oceanic populations.

Limits to the accurate and generalizable use of soundscapes to monitor biodiversity.

Although eco-acoustic monitoring has the potential to deliver biodiversity insight on vast scales, existing analytical approaches behave unpredictably across studies. We collated 8,023 audio recordings with paired manual avifaunal point counts to investigate whether soundscapes could be used to monitor biodiversity across diverse ecosystems. We found that neither univariate indices nor machine learning models were predictive of species richness across datasets but soundscape change was consistently indicative of community change. Our findings indicate that there are no common features of biodiverse soundscapes and that soundscape monitoring should be used cautiously and in conjunction with more reliable in-person ecological surveys.

Using deep learning to track time × frequency whistle contours of toothed whales without human-annotated training data.

Many odontocetes produce whistles that feature characteristic contour shapes in spectrogram representations of their calls. Automatically extracting the time × frequency tracks of whistle contours has numerous subsequent applications, including species classification, identification, and density estimation. Deep-learning-based methods, which train models using analyst-annotated whistles, offer a promising way to reliably extract whistle contours. However, the application of such methods can be limited by the significant amount of time and labor required for analyst annotation. To overcome this challenge, a technique that learns from automatically generated pseudo-labels has been developed. These annotations are less accurate than those generated by human analysts but more cost-effective to generate. It is shown that standard training methods do not learn effective models from these pseudo-labels. An improved loss function designed to compensate for pseudo-label error that significantly increases whistle extraction performance is introduced. The experiments show that the developed technique performs well when trained with pseudo-labels generated by two different algorithms. Models trained with the generated pseudo-labels can extract whistles with an F1-score (the harmonic mean of precision and recall) of 86.31% and 87.2% for the two sets of pseudo-labels that are considered. This performance is competitive with a model trained with 12 539 expert-annotated whistles (F1-score of 87.47%).

Ecological inferences about marine mammals from passive acoustic data.

Monitoring on the basis of sound recordings, or passive acoustic monitoring, can complement or serve as an alternative to real-time visual or aural monitoring of marine mammals and other animals by human observers. Passive acoustic data can support the estimation of common, individual-level ecological metrics, such as presence, detection-weighted occupancy, abundance and density, population viability and structure, and behaviour. Passive acoustic data also can support estimation of some community-level metrics, such as species richness and composition. The feasibility of estimation and certainty of estimates is highly context dependent, and understanding the factors that affect the reliability of measurements is useful for those considering whether to use passive acoustic data. Here, we review basic concepts and methods of passive acoustic sampling in marine systems that often are applicable to marine mammal research and conservation. Our ultimate aim is to facilitate collaboration among ecologists, bioacousticians, and data analysts. Ecological applications of passive acoustics require one to make decisions about sampling design, which in turn requires consideration of sound propagation, sampling of signals, and data storage. One also must make decisions about signal detection and classification and evaluation of the performance of algorithms for these tasks. Investment in the research and development of systems that automate detection and classification, including machine learning, are increasing. Passive acoustic monitoring is more reliable for detection of species presence than for estimation of other species-level metrics. Use of passive acoustic monitoring to distinguish among individual animals remains difficult. However, information about detection probability, vocalisation or cue rate, and relations between vocalisations and the number and behaviour of animals increases the feasibility of estimating abundance or density. Most sensor deployments are fixed in space or are sporadic, making temporal turnover in species composition more tractable to estimate than spatial turnover. Collaborations between acousticians and ecologists are most likely to be successful and rewarding when all partners critically examine and share a fundamental understanding of the target variables, sampling process, and analytical methods.

Environmental conditions and marine heatwaves influence blue whale foraging and reproductive effort.

Animal behavior is motivated by the fundamental need to feed and reproduce, and these behaviors can be inferred from spatiotemporal variations in biological signals such as vocalizations. Yet, linking foraging and reproductive effort to environmental drivers can be challenging for wide-ranging predator species. Blue whales are acoustically active marine predators that produce two distinct vocalizations: song and D calls. We examined environmental correlates of these vocalizations using continuous recordings from five hydrophones in the South Taranaki Bight region of Aotearoa New Zealand to investigate call behavior relative to ocean conditions and infer life history patterns. D calls were strongly correlated with oceanographic drivers of upwelling in spring and summer, indicating associations with foraging effort. In contrast, song displayed a highly seasonal pattern with peak intensity in fall, which aligned with the timing of conception inferred from whaling records. Finally, during a marine heatwave, reduced foraging (inferred from D calls) was followed by lower reproductive effort (inferred from song intensity).

Changes in Cough Airflow and Acoustics After Injection Laryngoplasty.

OBJECTIVE/HYPOTHESIS: We explored the following hypotheses in a cohort of patients undergoing injection laryngoplasty: (1) glottic insufficiency affects voluntary cough airflow dynamics and restoring glottic competence may improve parameters of cough strength, (2) cough strength can be inferred from cough acoustic signal, and (3) glottic competence changes cough sounds and correlates with spectrogram morphology. STUDY TYPE/DESIGN: Prospective interventional study. METHODS: Subjects with glottic insufficiency secondary to unilateral vocal fold paresis, paralysis, or atrophy, and scheduled for injection laryngoplasty completed an instrumental assessment of voluntary cough airflow using a pneumotachometer and a protocolized voluntary cough sound recording. A Wilcoxon signed-rank test was used to compare the differences between pre- and post-injection laryngoplasty in airflow and acoustic measures. A Spearman rank-order correlation was used to evaluate the association between airflow and acoustic cough measures. RESULTS: Twenty-five patients (13F:12M, mean age 68.8) completed voluntary cough airflow measurements and 22 completed cough sound recordings. Following injection laryngoplasty, patients had a statistically significant decreased peak expiratory flow rise time (PEFRT) (mean change: -0.03 s, SD: 0.06, p = 0.04) and increased cough volume acceleration (mean change: 13.1 L/s2 , SD: 33.9, p = 0.03), suggesting improved cough effectiveness. Correlation of cough acoustic measures with airflow measures showed a weak relationship between PEFRT and acoustic energy (coefficient: -0.31, p = 0.04) and peak power density (coefficient: -0.35, p = 0.02). CONCLUSIONS: Our study thus indicates that injection laryngoplasty may help avert aspiration in patients with glottic insufficiency by improving cough effectiveness and that improved cough airflow measures may be tracked with cough sounds. LEVEL OF EVIDENCE: 3 Laryngoscope, 133:S1-S14, 2023.

Acoustic Screening of the "Wet voice": Proof of Concept in an ex vivo Canine Laryngeal Model.

BACKGROUND: Current protocols for bedside swallow evaluation have high rates of false negative results. Though experts are not consistently able to screen for aspiration risk by assessing vocal quality, there is emerging evidence that vocal acoustic parameters are significantly different in patients at risk of aspiration. Herein, we aimed to determine whether the presence of material on the vocal folds in an excised canine laryngeal model may have an impact on acoustic and aerodynamic measures. METHODS: Two ex vivo canine larynges were tested. Three liquids of different viscosities (1:100 diluted glycerin, pure glycerin, and honey-thick Varibar) were placed on the vocal folds at a constant volume. Acoustic and aerodynamic measures were obtained in both adducted and abducted vocal fold configurations. Intraglottal high-speed imaging was used to approximate the maximum divergence angle of the larynges in the studied conditions and examine its relationship to vocal efficiency (VE) and acoustic measures. RESULTS: In glottic insufficiency conditions only, we found that several acoustic parameters could predict the presence of material on the vocal folds. Based on the combination of the aerodynamic and acoustic data, we found that decreased spectral energy in the higher harmonics was associated with decreased VE in the presence of material on the vocal folds and/or glottic insufficiency. CONCLUSION: Decreased spectral energy in the higher harmonics of the voice was found to be a potential biomarker of swallowing dysfunction, as it correlates with decreased vocal efficiency due to material on the vocal folds and/or glottic insufficiency, both of which are known risk factors for aspiration. LEVEL OF EVIDENCE: NA Laryngoscope, 133:2517-2524, 2023.

Silbido profundo: An open source package for the use of deep learning to detect odontocete whistles.

This work presents an open-source matlab software package for exploiting recent advances in extracting tonal signals from large acoustic data sets. A whistle extraction algorithm published by Li, Liu, Palmer, Fleishman, Gillespie, Nosal, Shiu, Klinck, Cholewiak, Helble, and Roch [(2020). Proceedings of the International Joint Conference on Neural Networks, July 19-24, Glasgow, Scotland, p. 10] is incorporated into silbido, an established software package for extraction of cetacean tonal calls. The precision and recall of the new system were over 96% and nearly 80%, respectively, when applied to a whistle extraction task on a challenging two-species subset of a conference-benchmark data set. A second data set was examined to assess whether the algorithm generalized to data that were collected across different recording devices and locations. These data included 487 h of weakly labeled, towed array data collected in the Pacific Ocean on two National Oceanographic and Atmospheric Administration (NOAA) cruises. Labels for these data consisted of regions of toothed whale presence for at least 15 species that were based on visual and acoustic observations and not limited to whistles. Although the lack of per whistle-level annotations prevented measurement of precision and recall, there was strong concurrence of automatic detections and the NOAA annotations, suggesting that the algorithm generalizes well to new data.

Detection probability and density estimation of fin whales by a Seaglider.

A single-hydrophone ocean glider was deployed within a cabled hydrophone array to demonstrate a framework for estimating population density of fin whales (Balaenoptera physalus) from a passive acoustic glider. The array was used to estimate tracks of acoustically active whales. These tracks became detection trials to model the detection function for glider-recorded 360-s windows containing fin whale 20-Hz pulses using a generalized additive model. Detection probability was dependent on both horizontal distance and low-frequency glider flow noise. At the median 40-Hz spectral level of 97 dB re 1 µPa2/Hz, detection probability was near one at horizontal distance zero with an effective detection radius of 17.1 km [coefficient of variation (CV) = 0.13]. Using estimates of acoustic availability and acoustically active group size from tagged and tracked fin whales, respectively, density of fin whales was estimated as 1.8 whales per 1000 km2 (CV = 0.55). A plot sampling density estimate for the same area and time, estimated from array data alone, was 1.3 whales per 1000 km2 (CV = 0.51). While the presented density estimates are from a small demonstration experiment and should be used with caution, the framework presented here advances our understanding of the potential use of gliders for cetacean density estimation.

Source separation with an acoustic vector sensor for terrestrial bioacoustics.

Passive acoustic monitoring is emerging as a low-cost, non-invasive methodology for automated species-level population surveys. However, systems for automating the detection and classification of vocalizations in complex soundscapes are significantly hindered by the overlap of calls and environmental noise. We propose addressing this challenge by utilizing an acoustic vector sensor to separate contributions from different sound sources. More specifically, we describe and implement an analytical pipeline consisting of (1) calculating direction-of-arrival, (2) decomposing the azimuth estimates into angular distributions for individual sources, and (3) numerically reconstructing source signals. Using both simulation and experimental recordings, we evaluate the accuracy of direction-of-arrival estimation through the active intensity method (AIM) against the baselines of white noise gain constraint beamforming (WNC) and multiple signal classification (MUSIC). Additionally, we demonstrate and compare source signal reconstruction with simple angular thresholding and a wrapped Gaussian mixture model. Overall, we show that AIM achieves higher performance than WNC and MUSIC, with a mean angular error of about 5°, robustness to environmental noise, flexible representation of multiple sources, and high fidelity in source signal reconstructions.

Shaken, not stirred: blue whales show no acoustic response to earthquake events.

Quantifying how animals respond to disturbance events bears relevance for understanding consequences to population health. We investigate whether blue whales respond acoustically to naturally occurring episodic noise by examining calling before and after earthquakes (27 040 calls, 32 earthquakes; 27 January-29 June 2016). Two vocalization types were evaluated: New Zealand blue whale song and downswept vocalizations ('D calls'). Blue whales did not alter the number of D calls, D call received level or song intensity following earthquakes (paired t-tests, p > 0.7 for all). Linear models accounting for earthquake strength and proximity revealed significant relationships between change in calling activity surrounding earthquakes and prior calling activity (D calls: R 2 = 0.277, p < 0.0001; song: R 2 = 0.080, p = 0.028); however, these same relationships were true for 'null' periods without earthquakes (D calls: R 2 = 0.262, p < 0.0001; song: R 2 = 0.149, p = 0.0002), indicating that the pattern is driven by blue whale calling context regardless of earthquake presence. Our findings that blue whales do not respond to episodic natural noise provide context for interpreting documented acoustic responses to anthropogenic noise sources, including shipping traffic and petroleum development, indicating that they potentially evolved tolerance for natural noise sources but not novel noise from anthropogenic origins.

The machine learning-powered BirdNET App reduces barriers to global bird research by enabling citizen science participation.

The BirdNET App, a free bird sound identification app for Android and iOS that includes over 3,000 bird species, reduces barriers to citizen science while generating tens of millions of bird observations globally that can be used to replicate known patterns in avian ecology.

Perspectives in machine learning for wildlife conservation.

Inexpensive and accessible sensors are accelerating data acquisition in animal ecology. These technologies hold great potential for large-scale ecological understanding, but are limited by current processing approaches which inefficiently distill data into relevant information. We argue that animal ecologists can capitalize on large datasets generated by modern sensors by combining machine learning approaches with domain knowledge. Incorporating machine learning into ecological workflows could improve inputs for ecological models and lead to integrated hybrid modeling tools. This approach will require close interdisciplinary collaboration to ensure the quality of novel approaches and train a new generation of data scientists in ecology and conservation.

Accounting for the Lombard effect in estimating the probability of detection in passive acoustic surveys: Applications for single sensor mitigation and monitoring.

The detection range of calling animals is commonly described by the passive sonar equations. However, the sonar equations do not account for interactions between source and ambient sound level, i.e., the Lombard effect. This behavior has the potential to introduce non-linearities into the sonar equations and result in incorrectly predicted detection ranges. Here, we investigate the relationship between ambient sound and effective detection ranges for North Atlantic right whales (Eubalaena glacialis) in Cape Cod Bay, MA, USA using a sparse array of acoustic recorders. Generalized estimating equations were used to model the probability that a call was detected as a function of distance between the calling animal and the sensor and the ambient sound level. The model suggests a non-linear relationship between ambient sound levels and the probability of detecting a call. Comparing the non-linear model to the linearized version of the same model resulted in 12 to 25% increases in the effective detection range. We also found evidence of the Lombard effect suggesting that it is the most plausible cause for the non-linearity in the relationship. Finally, we suggest a simple modification to the sonar equation for estimating detection probability for single sensor monitoring applications.

Improve automatic detection of animal call sequences with temporal context.

Many animals rely on long-form communication, in the form of songs, for vital functions such as mate attraction and territorial defence. We explored the prospect of improving automatic recognition performance by using the temporal context inherent in song. The ability to accurately detect sequences of calls has implications for conservation and biological studies. We show that the performance of a convolutional neural network (CNN), designed to detect song notes (calls) in short-duration audio segments, can be improved by combining it with a recurrent network designed to process sequences of learned representations from the CNN on a longer time scale. The combined system of independently trained CNN and long short-term memory (LSTM) network models exploits the temporal patterns between song notes. We demonstrate the technique using recordings of fin whale (Balaenoptera physalus) songs, which comprise patterned sequences of characteristic notes. We evaluated several variants of the CNN + LSTM network. Relative to the baseline CNN model, the CNN + LSTM models reduced performance variance, offering a 9-17% increase in area under the precision-recall curve and a 9-18% increase in peak F1-scores. These results show that the inclusion of temporal information may offer a valuable pathway for improving the automatic recognition and transcription of wildlife recordings.

Temporal and spatial lags between wind, coastal upwelling, and blue whale occurrence.

Understanding relationships between physical drivers and biological response is central to advancing ecological knowledge. Wind is the physical forcing mechanism in coastal upwelling systems, however lags between wind input and biological responses are seldom quantified for marine predators. Lags were examined between wind at an upwelling source, decreased temperatures along the upwelling plume's trajectory, and blue whale occurrence in New Zealand's South Taranaki Bight region (STB). Wind speed and sea surface temperature (SST) were extracted for austral spring-summer months between 2009 and 2019. A hydrophone recorded blue whale vocalizations October 2016-March 2017. Timeseries cross-correlation analyses were conducted between wind speed, SST at different locations along the upwelling plume, and blue whale downswept vocalizations (D calls). Results document increasing lag times (0-2 weeks) between wind speed and SST consistent with the spatial progression of upwelling, culminating with increased D call density at the distal end of the plume three weeks after increased wind speeds at the upwelling source. Lag between wind events and blue whale aggregations (n = 34 aggregations 2013-2019) was 2.09 ± 0.43 weeks. Variation in lag was significantly related to the amount of wind over the preceding 30 days, which likely influences stratification. This study enhances knowledge of physical-biological coupling in upwelling ecosystems and enables improved forecasting of species distribution patterns for dynamic management.