Bridging auditory perception and natural language processing with semantically informed deep neural networks.

ABSTRACT: Sound recognition is effortless for humans but poses a significant challenge for artificial hearing systems. Deep neural networks (DNNs), especially convolutional neural networks (CNNs), have recently surpassed traditional machine learning in sound classification. However, current DNNs map sounds to labels using binary categorical variables, neglecting the semantic relations between labels. Cognitive neuroscience research suggests that human listeners exploit such semantic information besides acoustic cues. Hence, our hypothesis is that incorporating semantic information improves DNN's sound recognition performance, emulating human behaviour. In our approach, sound recognition is framed as a regression problem, with CNNs trained to map spectrograms to continuous semantic representations fr

SUBMITTER: Esposito M 

PROVIDER: S-EPMC11383964 | biostudies-literature | 2024 Sep

REPOSITORIES: biostudies-literature