Comparative evaluation of large language models in delivering guideline-compliant recommendations for topical NSAID use in musculoskeletal pain: a multidimensional analysis.

ABSTRACT:

Introduction

While large language models (LLMs) are increasingly used in clinical decision support, their adherence to evidence-based guidelines-particularly for musculoskeletal pain management-remains understudied.

Methods

Four LLMs (DeepSeek-R1, ChatGPT-4o, Gemini, Grok-3) were evaluated on their responses to topical NSAID use for musculoskeletal pain through: assessments of response quality (accuracy, over-conclusiveness, supplementary information, and incompleteness), standardized readability metrics (Flesch Reading Ease, Flesch-Kincaid Grade Level), and the PEMAT-P tool to quantify actionability.

Results

The four LLMs showed significant variability in accuracy (ANOVA p = 0.045), with Gemini scoring highest (8.33 ± 0.77) and DeepSeek-R1 lowest (7.72 ± 1.52) and in

SUBMITTER: Dong C 

PROVIDER: S-EPMC12568900 | biostudies-literature | 2025 Nov

REPOSITORIES: biostudies-literature