Nonnegative matrix factorization with Wasserstein metric-based regularization for enhanced text embedding.

ABSTRACT: Text embedding plays a crucial role in natural language processing (NLP). Among various approaches, nonnegative matrix factorization (NMF) is an effective method for this purpose. However, the standard NMF approach, fundamentally based on the bag-of-words model, fails to utilize the contextual information of documents and may result in a significant loss of semantics. To address this issue, we propose a new NMF scheme incorporating a regularization term based on the Wasserstein metric, which quantifies an approximation of a word-context matrix to leverage semantic information. Since the word-context matrix is typically symmetric and positive definite (SPD), the Wasserstein metric used in the regularization is related to a natural SPD manifold structure. We then build upon this manifold str

SUBMITTER: Li M 

PROVIDER: S-EPMC11620366 | biostudies-literature | 2024

REPOSITORIES: biostudies-literature