Human researchers are superior to large language models in writing a medical systematic review in a comparative multitask assessment.

ABSTRACT: The capability of Large Language Models (LLMs) to support and facilitate research activities has sparked growing interest in their integration into scientific workflows. This paper aims to evaluate and compare against human researchers the performance of 6 different LLMs in conducting the various tasks necessary to produce a systematic literature review. The evaluation of the 6 LLMs was split into 3 tasks: literature search, article screening and selection (task 1); data extraction and analysis (task 2); final paper drafting (task 3). Their results were compared with a human-produced systematic review on the same topic, serving as reference standard. The evaluation was repeated on two rounds to evaluate between-version changes and improvements of LLMs over time. Out of the 18 scientific ar

SUBMITTER: Sollini M 

PROVIDER: S-EPMC12765003 | biostudies-literature | 2025 Dec

REPOSITORIES: biostudies-literature