biostudies-literatureLin ZMulticenter Investigator-Initiated Trial Project of Mianyang Central Hospital (Project Number: 2024KY-FZC033),Project of the Mianyang Municipal Health Commission(Grant No.202207),Incubation Project of Mianyang Central Hospital (Grant No.2019FH14)213https://www.ebi.ac.uk/biostudies/studies/S-EPMC12181206biostudies-literatureUnknown25(1)This study aimed to compare the performance of three large language models (ChatGPT-4o, OpenAI O1, and OpenAI O3 mini) in delivering accurate and guideline compliant recommendations for pneumonia management. By assessing both general and guideline-focused questions, the investigation sought to elucidate each model's strengths, limitations, and capacity to self-correct in response to expert feedback. Fifty pneumonia-related questions (30 general, 20 guideline-based) were posed to the three models. Ten infectious disease specialists independently scored responses for accuracy using a 5-point scale. The two chain-of-thought models (OpenAI O1 and OpenAI O3 mini) were further tested for self-correction when initially rated "poor," with re-evaluations conducted one week later to reduce recall biClinical and experimental medicinePerformance analysis of large language models Chatgpt-4o, OpenAI O1, and OpenAI O3 mini in clinical treatment of pneumonia: a comparative study.PMC121812062019FH14Liu HLi YWu MYu QXie JXiao GLin ZSong XfalsePerformance analysis of large language models Chatgpt-4o, OpenAI O1, and OpenAI O3 mini in clinical treatment of pneumonia: a comparative study.This study aimed to compare the performance of three large language models (ChatGPT-4o, OpenAI O1, and OpenAI O3 mini) in delivering accurate and guideline compliant recommendations for pneumonia management. By assessing both general and guideline-focused questions, the investigation sought to elucidate each model's strengths, limitations, and capacity to self-correct in response to expert feedback. Fifty pneumonia-related questions (30 general, 20 guideline-based) were posed to the three models. Ten infectious disease specialists independently scored responses for accuracy using a 5-point scale. The two chain-of-thought models (OpenAI O1 and OpenAI O3 mini) were further tested for self-correction when initially rated "poor," with re-evaluations conducted one week later to reduce recall bi2025-01-01T00:00:00Z2025 Jun2026-06-05T23:02:51.143Z2026-05-23T03:10:45.468ZS-EPMC121812064054009210.1007/s10238-025-01743-7