Evaluating the performance of ChatGPT-4 on the United Kingdom Medical Licensing Assessment.

ABSTRACT:

Introduction

Recent developments in artificial intelligence large language models (LLMs), such as ChatGPT, have allowed for the understanding and generation of human-like text. Studies have found LLMs abilities to perform well in various examinations including law, business and medicine. This study aims to evaluate the performance of ChatGPT in the United Kingdom Medical Licensing Assessment (UKMLA).

Methods

Two publicly available UKMLA papers consisting of 200 single-best-answer (SBA) questions were screened. Nine SBAs were omitted as they contained images that were not suitable for input. Each question was assigned a specialty based on the UKMLA content map published by the General Medical Council. A total of 191 SBAs were inputted in ChatGPT-4 through three attempts over

SUBMITTER: Lai UH 

PROVIDER: S-EPMC10547055 | biostudies-literature | 2023

REPOSITORIES: biostudies-literature