Assessing large language models' accuracy in providing patient support for choroidal melanoma.

ABSTRACT:

Purpose

This study aimed to evaluate the accuracy of information that patients can obtain from large language models (LLMs) when seeking answers to common questions about choroidal melanoma.

Methods

Comparative study comparing frequently asked questions from choroidal melanoma patients and queried three major LLMs-ChatGPT 3.5, Bing AI, and DocsGPT. Answers were reviewed by three ocular oncology experts and scored as accurate, partially accurate, or inaccurate. Statistical analysis compared the quality of responses across models.

Results

For medical advice questions, ChatGPT gave 92% accurate responses compared to 58% for Bing AI and DocsGPT. For pre/post-op questions, ChatGPT and Bing AI were 86% accurate while DocsGPT was 73% accurate. There were no statistically sig

SUBMITTER: Anguita R 

PROVIDER: S-EPMC11544095 | biostudies-literature | 2024 Nov

REPOSITORIES: biostudies-literature