Collaborative Large Language Models for Automated Data Extraction in Living Systematic Reviews.

ABSTRACT:

Objective

Data extraction from the published literature is the most laborious step in conducting living systematic reviews (LSRs). We aim to build a generalizable, automated data extraction workflow leveraging large language models (LLMs) that mimics the real-world two-reviewer process.

Materials and methods

A dataset of 10 clinical trials (22 publications) from a published LSR was used, focusing on 23 variables related to trial, population, and outcomes data. The dataset was split into prompt development (n=5) and held-out test sets (n=17). GPT-4-turbo and Claude-3-Opus were used for data extraction. Responses from the two LLMs were compared for concordance. In instances with discordance, original responses from each LLM were provided to the other LLM for cross-critique. Ev

SUBMITTER: Khan MA 

PROVIDER: S-EPMC11469465 | biostudies-literature | 2024 Sep

REPOSITORIES: biostudies-literature