Comparative performance of large language models for patient-initiated ophthalmology consultations.

ABSTRACT:

Background

Large language models (LLMs) are increasingly accessed by lay users for medical advice. This study aims to conduct a comprehensive evaluation of the responses generated by five large language models.

Methods

We identified 31 ophthalmology-related questions most frequently raised by patients during routine consultations and subsequently elicited responses from five large language models: ChatGPT-4o, DeepSeek-V3, Doubao, Wenxin Yiyan 4.0 Turbo, and Qwen. A five-point likert scale was employed to assess each model across five domains: accuracy, logical consistency, coherence, safety, and content accessibility. Additionally, textual characteristics, including character, word, and sentence counts, were quantitatively analyzed.

Results

ChatGPT-4o and DeepSeek-V3

SUBMITTER: Huang M 

PROVIDER: S-EPMC12497754 | biostudies-literature | 2025

REPOSITORIES: biostudies-literature