Clinical Large Language Model Evaluation by Expert Review (CLEVER): Framework Development and Validation.

ABSTRACT:

Background

The proliferation of both general purpose and health care-specific large language models (LLMs) has intensified the challenge of effectively evaluating and comparing them. Data contamination plagues the validity of public benchmarks, self-preference distorts LLM-as-a-judge approaches, and there is a gap between the tasks used to test models and those used in clinical practice.

Objective

In response, we propose CLEVER (Clinical Large Language Model Evaluation-Expert Review), a methodology for blind, randomized, preference-based evaluation by practicing medical doctors on specific tasks.

Methods

We demonstrate the methodology by comparing GPT-4o (OpenAI) against 2 health care-specific LLMs, with 8 billion and 70 billion parameters, over 3 tasks: clinical text

SUBMITTER: Kocaman V 

PROVIDER: S-EPMC12677871 | biostudies-literature | 2025 Dec

REPOSITORIES: biostudies-literature