MedBot vs RealDoc: efficacy of large language modeling in physician-patient communication for rare diseases.

ABSTRACT:

Objectives

This study assesses the abilities of 2 large language models (LLMs), GPT-4 and BioMistral 7B, in responding to patient queries, particularly concerning rare diseases, and compares their performance with that of physicians.

Materials and methods

A total of 103 patient queries and corresponding physician answers were extracted from EXABO, a question-answering forum dedicated to rare respiratory diseases. The responses provided by physicians and generated by LLMs were ranked on a Likert scale by a panel of 4 experts based on 4 key quality criteria for health communication: correctness, comprehensibility, relevance, and empathy.

Results

The performance of generative pretrained transformer 4 (GPT-4) was significantly better than the performance of the physicians

SUBMITTER: Weber MT 

PROVIDER: S-EPMC12012358 | biostudies-literature | 2025 May

REPOSITORIES: biostudies-literature