Evaluating large language models for criterion-based grading from agreement to consistency.

ABSTRACT: This study evaluates the ability of large language models (LLMs) to deliver criterion-based grading and examines the impact of prompt engineering with detailed criteria on grading. Using well-established human benchmarks and quantitative analyses, we found that even free LLMs achieve criterion-based grading with a detailed understanding of the criteria, underscoring the importance of domain-specific understanding over model complexity. These findings highlight the potential of LLMs to deliver scalable educational feedback.

SUBMITTER: Zhang DW 

PROVIDER: S-EPMC11683144 | biostudies-literature | 2024 Dec

REPOSITORIES: biostudies-literature