MedAlign: A Clinician-Generated Dataset for Instruction Following with Electronic Medical Records.

ABSTRACT: The ability of large language models (LLMs) to follow natural language instructions with human-level fluency suggests many opportunities in healthcare to reduce administrative burden and improve quality of care. However, evaluating LLMs on realistic text generation tasks for healthcare remains challenging. Existing question answering datasets for electronic health record (EHR) data fail to capture the complexity of information needs and documentation burdens experienced by clinicians. To address these challenges, we introduce MedAlign, a benchmark dataset of 983 natural language instructions for EHR data. MedAlign is curated by 15 clinicians (7 specialities), includes clinician-written reference responses for 303 instructions, and provides 276 longitudinal EHRs for grounding instruction-re

SUBMITTER: Fleming SL 

PROVIDER: S-EPMC12826664 | biostudies-literature | 2024 Mar

REPOSITORIES: biostudies-literature