A Bimodal Emotion Recognition Approach through the Fusion of Electroencephalography and Facial Sequences.

ABSTRACT: In recent years, human-computer interaction (HCI) systems have become increasingly popular. Some of these systems demand particular approaches for discriminating actual emotions through the use of better multimodal methods. In this work, a deep canonical correlation analysis (DCCA) based multimodal emotion recognition method is presented through the fusion of electroencephalography (EEG) and facial video clips. A two-stage framework is implemented, where the first stage extracts relevant features for emotion recognition using a single modality, while the second stage merges the highly correlated features from the two modalities and performs classification. Convolutional neural network (CNN) based Resnet50 and 1D-CNN (1-Dimensional CNN) have been utilized to extract features from facial vid

SUBMITTER: Muhammad F 

PROVIDER: S-EPMC10000366 | biostudies-literature | 2023 Mar

REPOSITORIES: biostudies-literature