Speech-In-Noise Comprehension is Improved When Viewing a Deep-Neural-Network-Generated Talking Face.

ABSTRACT: Listening in a noisy environment is challenging, but many previous studies have demonstrated that comprehension of speech can be substantially improved by looking at the talker's face. We recently developed a deep neural network (DNN) based system that generates movies of a talking face from speech audio and a single face image. In this study, we aimed to quantify the benefits that such a system can bring to speech comprehension, especially in noise. The target speech audio was masked with signal to noise ratios of -9, -6, -3, and 0 dB and was presented to subjects in three audio-visual (AV) stimulus conditions: (1) synthesized AV: audio with the synthesized talking face movie; (2) natural AV: audio with the original movie from the corpus; and (3) audio-only: audio with a static image of t

SUBMITTER: Shan T 

PROVIDER: S-EPMC9677167 | biostudies-literature | 2022 Jan-Dec

REPOSITORIES: biostudies-literature