Evaluation Framework of Large Language Models in Medical Documentation: Development and Usability Study.

ABSTRACT:

Background

The advancement of large language models (LLMs) offers significant opportunities for health care, particularly in the generation of medical documentation. However, challenges related to ensuring the accuracy and reliability of LLM outputs, coupled with the absence of established quality standards, have raised concerns about their clinical application.

Objective

This study aimed to develop and validate an evaluation framework for assessing the accuracy and clinical applicability of LLM-generated emergency department (ED) records, aiming to enhance artificial intelligence integration in health care documentation.

Methods

We organized the Healthcare Prompt-a-thon, a competitive event designed to explore the capabilities of LLMs in generating accurate medical r

SUBMITTER: Seo J 

PROVIDER: S-EPMC11618017 | biostudies-literature | 2024 Nov

REPOSITORIES: biostudies-literature