Performance analysis of large language models Chatgpt-4o, OpenAI O1, and OpenAI O3 mini in clinical treatment of pneumonia: a comparative study.

ABSTRACT: This study aimed to compare the performance of three large language models (ChatGPT-4o, OpenAI O1, and OpenAI O3 mini) in delivering accurate and guideline compliant recommendations for pneumonia management. By assessing both general and guideline-focused questions, the investigation sought to elucidate each model's strengths, limitations, and capacity to self-correct in response to expert feedback. Fifty pneumonia-related questions (30 general, 20 guideline-based) were posed to the three models. Ten infectious disease specialists independently scored responses for accuracy using a 5-point scale. The two chain-of-thought models (OpenAI O1 and OpenAI O3 mini) were further tested for self-correction when initially rated "poor," with re-evaluations conducted one week later to reduce recall bi

SUBMITTER: Lin Z 

PROVIDER: S-EPMC12181206 | biostudies-literature | 2025 Jun

REPOSITORIES: biostudies-literature