Speech-Driven Facial Animations Improve Speech-in-Noise Comprehension of Humans.

ABSTRACT: Understanding speech becomes a demanding task when the environment is noisy. Comprehension of speech in noise can be substantially improved by looking at the speaker's face, and this audiovisual benefit is even more pronounced in people with hearing impairment. Recent advances in AI have allowed to synthesize photorealistic talking faces from a speech recording and a still image of a person's face in an end-to-end manner. However, it has remained unknown whether such facial animations improve speech-in-noise comprehension. Here we consider facial animations produced by a recently introduced generative adversarial network (GAN), and show that humans cannot distinguish between the synthesized and the natural videos. Importantly, we then show that the end-to-end synthesized videos significant

SUBMITTER: Varano E 

PROVIDER: S-EPMC8766421 | biostudies-literature | 2021

REPOSITORIES: biostudies-literature