Evaluation of large language models as a diagnostic aid for complex medical cases.

ABSTRACT:

Background

The use of large language models (LLM) has recently gained popularity in diverse areas, including answering questions posted by patients as well as medical professionals.

Objective

To evaluate the performance and limitations of LLMs in providing the correct diagnosis for a complex clinical case.

Design

Seventy-five consecutive clinical cases were selected from the Massachusetts General Hospital Case Records, and differential diagnoses were generated by OpenAI's GPT3.5 and 4 models.

Results

The mean number of diagnoses provided by the Massachusetts General Hospital case discussants was 16.77, by GPT3.5 30 and by GPT4 15.45 (p < 0.0001). GPT4 was more frequently able to list the correct diagnosis as first (22% versus 20% with GPT3.5, p = 

SUBMITTER: Rios-Hoyo A 

PROVIDER: S-EPMC11222590 | biostudies-literature | 2024

REPOSITORIES: biostudies-literature