LLM evaluation for thyroid nodule assessment: comparing ACR-TIRADS, C-TIRADS, and clinician-AI trust gap.

ABSTRACT:

Objective

To evaluate the diagnostic performance and clinical utility of advanced large language models (LLMs) -GPT-4o, GPT-o3-mini, and DeepSeek-R1- in stratifying thyroid nodule malignancy risk and generating guideline-aligned management recommendations based on structured narrative ultrasound descriptions.

Methods

This diagnostic modeling study evaluated three LLMs-GPT-4o, GPT-o3-mini, and DeepSeek-R1-using standardized narrative ultrasound descriptors. These descriptors were annotated by consensus among three senior board-certified sonologists and processed independently in a stateless manner to ensure unbiased outputs. LLM outputs were assessed under both ACR-TIRADS and C-TIRADS frameworks. Two experienced clinicians (a thyroid surgeon and an endocrinologist) independen

SUBMITTER: Dai X 

PROVIDER: S-EPMC12515656 | biostudies-literature | 2025

REPOSITORIES: biostudies-literature