Evaluation and comparison of large language models' responses to questions related optic neuritis.

ABSTRACT:

Objectives

Large language models (LLMs) show promise as clinical consultation tools and may assist optic neuritis patients, though research on their performance in this area is limited. Our study aims to assess and compare the performance of four commonly used LLM-Chatbots-Claude-2, ChatGPT-3.5, ChatGPT-4.0, and Google Bard-in addressing questions related to optic neuritis.

Methods

We curated 24 optic neuritis-related questions and had three ophthalmologists rate the responses on two three-point scales for accuracy and comprehensiveness. We also assessed readability using four scales. The final results showed performance differences among the four LLM-Chatbots.

Results

The average total accuracy scores (out of 9): ChatGPT-4.0 (7.62 ± 0.86), Google Bard (7.42 ± 1.20),

SUBMITTER: He HJ 

PROVIDER: S-EPMC12238082 | biostudies-literature | 2025

REPOSITORIES: biostudies-literature