Quantitative evaluation of GPT-4's performance on US and Chinese osteoarthritis treatment guideline interpretation and orthopaedic case consultation.

ABSTRACT:

Objectives

To evaluate GPT-4's performance in interpreting osteoarthritis (OA) treatment guidelines from the USA and China, and to assess its ability to diagnose and manage orthopaedic cases.

Setting

The study was conducted using publicly available OA treatment guidelines and simulated orthopaedic case scenarios.

Participants

No human participants were involved. The evaluation focused on GPT-4's responses to clinical guidelines and case questions, assessed by two orthopaedic specialists.

Outcomes

Primary outcomes included the accuracy and completeness of GPT-4's responses to guideline-based queries and case scenarios. Metrics included the correct match rate, completeness score and stratification of case responses into predefined tiers of correctness.

Results

SUBMITTER: Li J 

PROVIDER: S-EPMC11749315 | biostudies-literature | 2024 Dec

REPOSITORIES: biostudies-literature