A Performance Evaluation of Large Language Models in Keratoconus: A Comparative Study of ChatGPT-3.5, ChatGPT-4.0, Gemini, Copilot, Chatsonic, and Perplexity.

ABSTRACT: Background: This study evaluates the ability of six popular chatbots; ChatGPT-3.5, ChatGPT-4.0, Gemini, Copilot, Chatsonic, and Perplexity, to provide reliable answers to questions concerning keratoconus. Methods: Chatbots responses were assessed using mDISCERN (range: 15-75) and Global Quality Score (GQS) (range: 1-5) metrics. Readability was evaluated using nine validated readability assessments. We also addressed the quality and accountability of websites from which the questions originated. Results: We analyzed 20 websites, 65% "Private practice or independent user" and 35% "Official patient education materials". The mean JAMA benchmark score was 1.40 ± 0.91 (0-4 points), indicating low accountability. Reliability, measured using mDISCERN, ranged from 42.91 ± 3.15

SUBMITTER: Reyhan AH 

PROVIDER: S-EPMC11547000 | biostudies-literature | 2024 Oct

REPOSITORIES: biostudies-literature