Extracting Clinical Guideline Information Using Two Large Language Models: Evaluation Study.

ABSTRACT:

Background

The effective implementation of personalized pharmacogenomics (PGx) requires the integration of released clinical guidelines into decision support systems to facilitate clinical applications. Large language models (LLMs) can be valuable tools for automating information extraction and updates.

Objective

This study aimed to assess the effectiveness of repeated cross-comparisons and an agreement-threshold strategy in 2 advanced LLMs as supportive tools for updating information.

Methods

The study evaluated the performance of 2 LLMs, GPT-4o and Gemini-1.5-Pro, in extracting PGx clinical guidelines and comparing their outputs with expert-annotated evaluations. The 2 LLMs classified 385 PGx clinical guidelines, with each recommendation tested 20 times per model. A

SUBMITTER: Hsu HY 

PROVIDER: S-EPMC12413144 | biostudies-literature | 2025 Sep

REPOSITORIES: biostudies-literature