ChatGPT performance in assessing musculoskeletal MRI scan appropriateness based on ACR appropriateness criteria.

ABSTRACT: Large Language Models (LLMs) hold potential as clinical decision support tools, particularly when integrated with domain-specific knowledge. In radiology, there is limited research on LLMs for assessing imaging appropriateness. This study evaluates a contextualized GPT-4-based LLM's performance in assessing the appropriateness of musculoskeletal MRI scan requests with standard models and different versions of optimization. The LLMs' performances was also compared against human clinicians with varying experience (two radiology residents, two subspecialist attendings, an orthopaedic surgeon). Using a retrieval-augmented generation framework, the LLM was provided with a domain-specific knowledge base from 33 American College of Radiology Appropriateness Criteria guidelines. A test dataset of

SUBMITTER: Tan JR 

PROVIDER: S-EPMC11871060 | biostudies-literature | 2025 Feb

REPOSITORIES: biostudies-literature