Evaluating Large Language Models for Automated Reporting and Data Systems Categorization: Cross-Sectional Study.

ABSTRACT:

Background

Large language models show promise for improving radiology workflows, but their performance on structured radiological tasks such as Reporting and Data Systems (RADS) categorization remains unexplored.

Objective

This study aims to evaluate 3 large language model chatbots-Claude-2, GPT-3.5, and GPT-4-on assigning RADS categories to radiology reports and assess the impact of different prompting strategies.

Methods

This cross-sectional study compared 3 chatbots using 30 radiology reports (10 per RADS criteria), using a 3-level prompting strategy: zero-shot, few-shot, and guideline PDF-informed prompts. The cases were grounded in Liver Imaging Reporting & Data System (LI-RADS) version 2018, Lung CT (computed tomography) Screening Reporting & Data System (Lung-R

SUBMITTER: Wu Q 

PROVIDER: S-EPMC11292156 | biostudies-literature | 2024 Jul

REPOSITORIES: biostudies-literature