Natural sounds can be reconstructed from human neuroimaging data using deep neural network representation.

ABSTRACT: Reconstruction of perceptual experiences from brain activity offers a unique window into how population neural responses represent sensory information. Although decoding visual content from functional MRI (fMRI) has seen significant success, reconstructing arbitrary sounds remains challenging due to the fine temporal structure of auditory signals and the coarse temporal resolution of fMRI. Drawing on the hierarchical auditory features of deep neural networks (DNNs) with progressively larger time windows and their neural activity correspondence, we introduce a method for sound reconstruction that integrates brain decoding of DNN features and an audio-generative model. DNN features decoded from auditory cortical activity outperformed spectrotemporal and modulation-based features, enabling pe

SUBMITTER: Park JY 

PROVIDER: S-EPMC12313072 | biostudies-literature | 2025 Jul

REPOSITORIES: biostudies-literature