AI versus human-generated multiple-choice questions for medical education: a cohort study in a high-stakes examination.

ABSTRACT:

Background

The creation of high-quality multiple-choice questions (MCQs) is essential for medical education assessments but is resource-intensive and time-consuming when done by human experts. Large language models (LLMs) like ChatGPT-4o offer a promising alternative, but their efficacy remains unclear, particularly in high-stakes exams.

Objective

This study aimed to evaluate the quality and psychometric properties of ChatGPT-4o-generated MCQs compared to human-created MCQs in a high-stakes medical licensing exam.

Methods

A prospective cohort study was conducted among medical doctors preparing for the Primary Examination on Emergency Medicine (PEEM) organised by the Hong Kong College of Emergency Medicine in August 2024. Participants attempted two sets of 100 MCQs-one

SUBMITTER: Law AK 

PROVIDER: S-EPMC11806894 | biostudies-literature | 2025 Feb

REPOSITORIES: biostudies-literature