Development and evaluation of a 4M taxonomy from nursing home staff text messages using a fine-tuned generative language model.

ABSTRACT:

Objective

This study aimed to explore the utilization of a fine-tuned language model to extract expressions related to the Age-Friendly Health Systems 4M Framework (What Matters, Medication, Mentation, and Mobility) from nursing home worker text messages, deploy automated mapping of these expressions to a taxonomy, and explore the created expressions and relationships.

Materials and methods

The dataset included 21 357 text messages from healthcare workers in 12 Missouri nursing homes. A sample of 860 messages was annotated by clinical experts to form a "Gold Standard" dataset. Model performance was evaluated using classification metrics including Cohen's Kappa (κ), with κ ≥ 0.60 as the performance threshold. The selected model was fine-tuned. Extractions were clustered, labe

SUBMITTER: Farmer MS 

PROVIDER: S-EPMC11833468 | biostudies-literature | 2025 Mar

REPOSITORIES: biostudies-literature