Evaluation of large language models for antimicrobial classification: implications for antimicrobial stewardship programs.

ABSTRACT:

Objective

To evaluate the ability of large language models (LLMs) with targeted feedback to classify medications as antimicrobial or non-antimicrobial and their implication in antimicrobial stewardship.

Design

Cross-sectional evaluation using a two-phase process: initial unguided classification and feedback-informed reclassification.

Setting

Medication-level analysis of health system prescribing data.

Participants

A data set of 7,239 unique medication entries from health systems in the Collaboration to Harmonize Antimicrobial Registry Measures (CHARM) project.

Methods

Four LLMs, including ChatGPT-3.5, Copilot GPT-4o, Claude Sonnet 4, and Gemini 2.5 Flash, classified all entries against a manual reference standard. Models then received feedback on 20% of

SUBMITTER: Vo T 

PROVIDER: S-EPMC12722537 | biostudies-literature | 2025

REPOSITORIES: biostudies-literature