BioLORD-2023: semantic textual representations fusing large language models and clinical knowledge graph insights.

ABSTRACT:

Objective

In this study, we investigate the potential of large language models (LLMs) to complement biomedical knowledge graphs in the training of semantic models for the biomedical and clinical domains.

Materials and methods

Drawing on the wealth of the Unified Medical Language System knowledge graph and harnessing cutting-edge LLMs, we propose a new state-of-the-art approach for obtaining high-fidelity representations of biomedical concepts and sentences, consisting of 3 steps: an improved contrastive learning phase, a novel self-distillation phase, and a weight averaging phase.

Results

Through rigorous evaluations of diverse downstream tasks, we demonstrate consistent and substantial improvements over the previous state of the art for semantic textual similarity (S

SUBMITTER: Remy F 

PROVIDER: S-EPMC11339519 | biostudies-literature | 2024 Sep

REPOSITORIES: biostudies-literature