Development and evaluation of prompts for a large language model to screen titles and abstracts in a living systematic review.

ABSTRACT:

Background

Living systematic reviews (LSRs) maintain an updated summary of evidence by incorporating newly published research. While they improve review currency, repeated screening and selection of new references make them labourious and difficult to maintain. Large language models (LLMs) show promise in assisting with screening and data extraction, but more work is needed to achieve the high accuracy required for evidence that informs clinical and policy decisions.

Objective

The study evaluated the effectiveness of an LLM (GPT-4o) in title and abstract screening compared with human reviewers.

Methods

Human decisions from an LSR on prodopaminergic interventions for anhedonia served as the reference standard. The baseline search results were divided into a development

SUBMITTER: Homiar A 

PROVIDER: S-EPMC12306261 | biostudies-literature | 2025 Jul

REPOSITORIES: biostudies-literature