Leveraging open-source large language models for clinical information extraction in resource-constrained settings.

ABSTRACT:

Objective

We aimed to evaluate the zero-shot performance of open-source generative large language models (LLMs) on clinical information extraction from Dutch medical reports using the Diagnostic Report Analysis: General Optimization of NLP (DRAGON) benchmark.

Methods

We developed and released the llm_extractinator framework, a scalable, open-source tool for automating information extraction from clinical texts using LLMs. We evaluated 9 multilingual open-source LLMs across 28 tasks in the DRAGON benchmark, covering classification, regression, and named entity recognition (NER). All tasks were performed in a zero-shot setting. Model performance was quantified using task-specific metrics and aggregated into a DRAGON utility score. Additionally, we investigated the effect of in

SUBMITTER: Builtjes L 

PROVIDER: S-EPMC12488231 | biostudies-literature | 2025 Oct

REPOSITORIES: biostudies-literature