Assessing ChatGPT's Capability for Multiple Choice Questions Using RaschOnline: Observational Study.

ABSTRACT:

Background

ChatGPT (OpenAI), a state-of-the-art large language model, has exhibited remarkable performance in various specialized applications. Despite the growing popularity and efficacy of artificial intelligence, there is a scarcity of studies that assess ChatGPT's competence in addressing multiple-choice questions (MCQs) using KIDMAP of Rasch analysis-a website tool used to evaluate ChatGPT's performance in MCQ answering.

Objective

This study aims to (1) showcase the utility of the website (Rasch analysis, specifically RaschOnline), and (2) determine the grade achieved by ChatGPT when compared to a normal sample.

Methods

The capability of ChatGPT was evaluated using 10 items from the English tests conducted for Taiwan college entrance examinations in 2023. Under a

SUBMITTER: Chow JC 

PROVIDER: S-EPMC11346125 | biostudies-literature | 2024 Aug

REPOSITORIES: biostudies-literature