Improving medication error classification using a reasoning large language model.

ABSTRACT:

Objectives

To assess the performance of a reasoning large language model (LLM) in identifying medication errors in medical incident reports.

Materials and methods

OpenAI's O4-mini LLM was adapted using prompt engineering on 75 000 anonymized incident reports from the Västmanland region of Sweden (2019-2024). To guide the prompt design, we used a subset of 2434 reports, which were manually reclassified by pharmacists as medication-related or not. For validation, 200 reports (January 2024-March 2024) were independently classified by 2 pharmacists to establish a reference classification. Moreover, the LLM performed binary classification, with concordance rates measured against the expert consensus.

Results

The LLM achieved a concordance rate of 96.0% (192/200; 95% CI, 92

SUBMITTER: Krifors A 

PROVIDER: S-EPMC12832951 | biostudies-literature | 2026 Feb

REPOSITORIES: biostudies-literature