Zero-Shot PI-RADS Version 2.1 Scoring with ChatGPT-4 Turbo and Llama 3: Diagnostic Performance and Agreement with Abdominal Radiologists.

ABSTRACT: This retrospective, single-center study aimed to assess the diagnostic performance and agreement of two large language models (LLMs), ChatGPT-4 Turbo (OpenAI) and Llama 3 (Meta AI), in assigning Prostate Imaging Reporting and Data System (PI-RADS) scores to prostate MRI reports and to compare their performance with two abdominal radiologists. Structured prostate MRI reports (n = 500) obtained between January and December 2022, with original PI-RADS scores removed, were processed with LLMs using a standardized prompt to extract PI-RADS version 2.1 scores. Two abdominal radiologists independently assigned scores, with a third adjudicating discrepancies. Prostate biopsy results served as the reference standard for diagnostic performance assessment. There was high agreement between both

SUBMITTER: Firoozeh N 

PROVIDER: S-EPMC12862468 | biostudies-literature | 2026 Jan

REPOSITORIES: biostudies-literature