Reliability of ChatGPT in automated essay scoring for dental undergraduate examinations.

ABSTRACT:

Background

This study aimed to answer the research question: How reliable is ChatGPT in automated essay scoring (AES) for oral and maxillofacial surgery (OMS) examinations for dental undergraduate students compared to human assessors?

Methods

Sixty-nine undergraduate dental students participated in a closed-book examination comprising two essays at the National University of Singapore. Using pre-created assessment rubrics, three assessors independently performed manual essay scoring, while one separate assessor performed AES using ChatGPT (GPT-4). Data analyses were performed using the intraclass correlation coefficient and Cronbach's α to evaluate the reliability and inter-rater agreement of the test scores among all assessors. The mean scores of manual versus automated sco

SUBMITTER: Quah B 

PROVIDER: S-EPMC11373238 | biostudies-literature | 2024 Sep

REPOSITORIES: biostudies-literature