Dataset Information

Automated annotation of birdsong with a neural network that segments spectrograms.

ABSTRACT: Songbirds provide a powerful model system for studying sensory-motor learning. However, many analyses of birdsong require time-consuming, manual annotation of its elements, called syllables. Automated methods for annotation have been proposed, but these methods assume that audio can be cleanly segmented into syllables, or they require carefully tuning multiple statistical models. Here, we present TweetyNet: a single neural network model that learns how to segment spectrograms of birdsong into annotated syllables. We show that TweetyNet mitigates limitations of methods that rely on segmented audio. We also show that TweetyNet performs well across multiple individuals from two species of songbirds, Bengalese finches and canaries. Lastly, we demonstrate that using TweetyNet we can accurately annotate very large datasets containing multiple days of song, and that these predicted annotations replicate key findings from behavioral studies. In addition, we provide open-source software to assist other researchers, and a large dataset of annotated canary song that can serve as a benchmark. We conclude that TweetyNet makes it possible to address a wide range of new questions about birdsong.

SUBMITTER: Cohen Y

PROVIDER: S-EPMC8860439 | biostudies-literature | 2022 Jan

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Automated annotation of birdsong with a neural network that segments spectrograms.

Cohen Yarden Y Nicholson David Aaron DA Sanchioni Alexa A Mallaber Emily K EK Skidanova Viktoriya V Gardner Timothy J TJ

eLife 20220120

Songbirds provide a powerful model system for studying sensory-motor learning. However, many analyses of birdsong require time-consuming, manual annotation of its elements, called syllables. Automated methods for annotation have been proposed, but these methods assume that audio can be cleanly segmented into syllables, or they require carefully tuning multiple statistical models. Here, we present TweetyNet: a single neural network model that learns how to segment spectrograms of birdsong into an ...[more]

PMID: 35050849

Similar Datasets

Project description:The use of automatic acoustic recorders is becoming a principal method to survey birds in their natural habitats, as it is relatively noninvasive while still being informative. As with any other sound, birdsong degrades in amplitude, frequency, and temporal structure as it propagates to the recorder through the environment. Knowing how different birdsongs attenuate under different conditions is useful to, for example, develop protocols for deploying acoustic recorders and improve automated detection methods, an essential part of the research field that is becoming known as ecoacoustics. This article presents playback and recapture (record) experiments carried out under different environmental conditions using twenty bird calls from eleven New Zealand bird species in a native forest and an open area, answering five research questions: (1) How does birdsong attenuation differ between forest and open space? (2) What is the relationship between transmission height and birdsong attenuation? (3) How does frequency of birdsong impact the degradation of sound with distance? (4) Is birdsong attenuation different during the night compared to the day? and (5) what is the impact of wind on attenuation? Bird calls are complex sounds; therefore, we have chosen to use them rather than simple tones to ensure that this complexity is not missed in the analysis. The results demonstrate that birdsong transmission was significantly better in the forest than in the open site. During the night, the attenuation was at a minimum in both experimental sites. Transmission height affected the propagation of the songs of many species, particularly the flightless ones. The effect of wind was severe in the open site and attenuated lower frequencies. The reverberations due to reflective surfaces masked higher frequencies (8 kHz) in the forest even at moderate distances. The findings presented here can be applied to develop protocols for passive acoustic monitoring. Even though the attenuation can be generalized to frequency bands, the structure of the birdsong is also important. Selecting a reasonable sampling frequency avoids unnecessary data accumulation because higher frequencies attenuate more in the forest. Even at moderate distances, recorders capture significantly attenuated birdsong, and hence, automated analysis methods for field recordings need to be able to detect and recognize faint birdsong.

Project description:BackgroundAutomatic segmentation and localization of lesions in mammogram (MG) images are challenging even with employing advanced methods such as deep learning (DL) methods. We developed a new model based on the architecture of the semantic segmentation U-Net model to precisely segment mass lesions in MG images. The proposed end-to-end convolutional neural network (CNN) based model extracts contextual information by combining low-level and high-level features. We trained the proposed model using huge publicly available databases, (CBIS-DDSM, BCDR-01, and INbreast), and a private database from the University of Connecticut Health Center (UCHC).ResultsWe compared the performance of the proposed model with those of the state-of-the-art DL models including the fully convolutional network (FCN), SegNet, Dilated-Net, original U-Net, and Faster R-CNN models and the conventional region growing (RG) method. The proposed Vanilla U-Net model outperforms the Faster R-CNN model significantly in terms of the runtime and the Intersection over Union metric (IOU). Training with digitized film-based and fully digitized MG images, the proposed Vanilla U-Net model achieves a mean test accuracy of 92.6%. The proposed model achieves a mean Dice coefficient index (DI) of 0.951 and a mean IOU of 0.909 that show how close the output segments are to the corresponding lesions in the ground truth maps. Data augmentation has been very effective in our experiments resulting in an increase in the mean DI and the mean IOU from 0.922 to 0.951 and 0.856 to 0.909, respectively.ConclusionsThe proposed Vanilla U-Net based model can be used for precise segmentation of masses in MG images. This is because the segmentation process incorporates more multi-scale spatial context, and captures more local and global context to predict a precise pixel-wise segmentation map of an input full MG image. These detected maps can help radiologists in differentiating benign and malignant lesions depend on the lesion shapes. We show that using transfer learning, introducing augmentation, and modifying the architecture of the original model results in better performance in terms of the mean accuracy, the mean DI, and the mean IOU in detecting mass lesion compared to the other DL and the conventional models.

Dataset Information

Automated annotation of birdsong with a neural network that segments spectrograms.

Publications

Automated annotation of birdsong with a neural network that segments spectrograms.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets