Real-time multilingual speech recognition and speaker diarization system based on Whisper segmentation.

ABSTRACT: This research presents the development of a cutting-edge real-time multilingual speech recognition and speaker diarization system that leverages OpenAI's Whisper model. The system specifically addresses the challenges of automatic speech recognition (ASR) and speaker diarization (SD) in dynamic, multispeaker environments, with a focus on accurately processing Mandarin speech with Taiwanese accents and managing frequent speaker switches. Traditional speech recognition systems often fall short in such complex multilingual and multispeaker contexts, particularly in SD. This study, therefore, integrates advanced speech recognition with speaker diarization techniques optimized for real-time applications. These optimizations include handling model outputs efficiently and incorporating speaker em

SUBMITTER: Lyu KM 

PROVIDER: S-EPMC11041969 | biostudies-literature | 2024

REPOSITORIES: biostudies-literature