Harnessing multimodal approaches for depression detection using large language models and facial expressions.

ABSTRACT: Detecting depression is a critical component of mental health diagnosis, and accurate assessment is essential for effective treatment. This study introduces a novel, fully automated approach to predicting depression severity using the E-DAIC dataset. We employ Large Language Models (LLMs) to extract depression-related indicators from interview transcripts, utilizing the Patient Health Questionnaire-8 (PHQ-8) score to train the prediction model. Additionally, facial data extracted from video frames is integrated with textual data to create a multimodal model for depression severity prediction. We evaluate three approaches: text-based features, facial features, and a combination of both. Our findings show the best results are achieved by enhancing text data with speech quality assessment, wi

SUBMITTER: Sadeghi M 

PROVIDER: S-EPMC11666580 | biostudies-literature | 2024 Dec

REPOSITORIES: biostudies-literature