Comparing Scoring Consistency of Large Language Models with Faculty for Formative Assessments in Medical Education.

ABSTRACT:

Background

The Liaison Committee on Medical Education requires that medical students receive individualized feedback on their self-directed learning skills. Pre-clinical students are asked to complete multiple spaced critical appraisal assignments. However, the individual feedback requires significant faculty time. As large language models (LLMs) can score and generate feedback, we explored their use in grading formative assessments through validity and feasibility lenses.

Objective

To explore the consistency and feasibility of using an LLM to assess and provide feedback for formative assessments in undergraduate medical education.

Design and participants

This was a cross-sectional study of pre-clinical students' critical appraisal assignments at University of Illinoi

SUBMITTER: Sreedhar R 

PROVIDER: S-EPMC11780228 | biostudies-literature | 2025 Jan

REPOSITORIES: biostudies-literature