Comparing large language models and human doctors in symptom-driven online medical consultations: A case study on trigeminal neuralgia.

ABSTRACT:

Objective

To evaluate the performance of generative AI tools, specifically Ernie Bot and ChatGPT, in supporting online medical consultations in China, focusing on their accuracy, safety, and empathy, and to assess their potential role in addressing the supply-demand gap in the healthcare system.

Methods

We collected 233 trigeminal neuralgia consultations from a Chinese medical platform, including patient questions and doctor replies. Each question was input into ChatGPT-3.5 and Ernie Bot with role-specific prompts to generate large language models (LLMs) responses. Four blinded raters-two doctors and two patients-evaluated all responses using DISCERN and a modified PEMAT. Lexical, syntactic, and semantic analyses were conducted, with Spearman correlations assessing links bet

SUBMITTER: Duan L 

PROVIDER: S-EPMC12559625 | biostudies-literature | 2025 Jan-Dec

REPOSITORIES: biostudies-literature