Structured matching models in multimodal information fusion: An optimized Kuhn-Munkres algorithm.

ABSTRACT: In modern multimodal interaction design, integrating information from diverse modalities-such as speech, vision, and text-presents a significant challenge. These modalities differ in structure, timing, and data volume, often leading to mismatches, low computational efficiency, and suboptimal user experiences during the integration process. This study aims to enhance both the efficiency and accuracy of multimodal information fusion. To achieve this, publicly available datasets-Carnegie Mellon University Multimodal Opinion Sentiment Intensity (CMU-MOSI) and Interactive Emotional Dyadic Motion Capture (IEMOCAP)-are employed to collect speech, visual, and textual data relevant to multimodal interaction scenarios. The data undergo preprocessing steps including noise reduction, feature extractio

SUBMITTER: Ji Q 

PROVIDER: S-EPMC12637911 | biostudies-literature | 2025

REPOSITORIES: biostudies-literature