A comparative study of large language model-based zero-shot inference and task-specific supervised classification of breast cancer pathology reports.

ABSTRACT:

Objective

Although supervised machine learning is popular for information extraction from clinical notes, creating large annotated datasets requires extensive domain expertise and is time-consuming. Meanwhile, large language models (LLMs) have demonstrated promising transfer learning capability. In this study, we explored whether recent LLMs could reduce the need for large-scale data annotations.

Materials and methods

We curated a dataset of 769 breast cancer pathology reports, manually labeled with 12 categories, to compare zero-shot classification capability of the following LLMs: GPT-4, GPT-3.5, Starling, and ClinicalCamel, with task-specific supervised classification performance of 3 models: random forests, long short-term memory networks with attention (LSTM-Att), and t

SUBMITTER: Sushil M 

PROVIDER: S-EPMC11413420 | biostudies-literature | 2024 Oct

REPOSITORIES: biostudies-literature