A data-efficient and easy-to-use lip language interface based on wearable motion capture and speech movement reconstruction.

ABSTRACT: Lip language recognition urgently needs wearable and easy-to-use interfaces for interference-free and high-fidelity lip-reading acquisition and to develop accompanying data-efficient decoder-modeling methods. Existing solutions suffer from unreliable lip reading, are data hungry, and exhibit poor generalization. Here, we propose a wearable lip language decoding technology that enables interference-free and high-fidelity acquisition of lip movements and data-efficient recognition of fluent lip language based on wearable motion capture and continuous lip speech movement reconstruction. The method allows us to artificially generate any wanted continuous speech datasets from a very limited corpus of word samples from users. By using these artificial datasets to train the decoder, we achieve an

SUBMITTER: Liu S 

PROVIDER: S-EPMC11204283 | biostudies-literature | 2024 Jun

REPOSITORIES: biostudies-literature