Testing the utility of GPT for title and abstract screening in environmental systematic evidence synthesis.

ABSTRACT: In this paper we show that OpenAI's Large Language Model (LLM) GPT perform remarkably well when used for title and abstract eligibility screening of scientific articles and within a (systematic) literature review workflow. We evaluated GPT on screening data from a systematic review study on electric vehicle charging infrastructure demand with almost 12,000 records using the same eligibility criteria as human screeners. We tested 3 different versions of this model that were tasked to distinguishing between relevant and irrelevant content by responding with a relevance probability between 0 and 1. For the latest GPT-4 model (tested in November 2023) and probability cutoff 0.5 the recall rate is 100%, meaning no relevant papers were missed and using this mode for screening would have saved 50

SUBMITTER: Nykvist B 

PROVIDER: S-EPMC12016299 | biostudies-literature | 2025 Apr

REPOSITORIES: biostudies-literature