Knowledge-Practice Performance Gap in Clinical Large Language Models: Systematic Review of 39 Benchmarks.

ABSTRACT:

Background

The evaluation of large language models (LLMs) in medicine has undergone a shift from knowledge-based testing to practice-based assessment, representing an evolution in how we measure artificial intelligence readiness for clinical deployment. While LLMs now routinely exceed human performance on medical licensing examinations, their translation to clinical practice remains poorly characterized.

Objective

This systematic review aims to categorize and analyze medical LLM benchmarks, examining performance patterns across different evaluation paradigms and identifying gaps in current assessment methodologies.

Methods

The protocol was registered at PROSPERO (CRD420251139729). Four databases (MEDLINE/PubMed, Embase/Ovid, Cochrane Library, and arXiv) were searched

SUBMITTER: Gong EJ 

PROVIDER: S-EPMC12706444 | biostudies-literature | 2025 Dec

REPOSITORIES: biostudies-literature