Unveiling GPT-4V's hidden challenges behind high accuracy on USMLE questions: Observational Study.

ABSTRACT:

Background

Recent advancements in artificial intelligence, such as GPT-3.5 Turbo (OpenAI) and GPT-4, have demonstrated significant potential by achieving good scores on text-only United States Medical Licensing Examination (USMLE) exams and effectively answering questions from physicians. However, the ability of these models to interpret medical images remains underexplored.

Objective

This study aimed to comprehensively evaluate the performance, interpretability, and limitations of GPT-3.5 Turbo, GPT-4, and its successor, GPT-4 Vision (GPT-4V), specifically focusing on GPT-4V's newly introduced image-understanding feature. By assessing the models on medical licensing examination questions that require image interpretation, we sought to highlight the strengths and weaknesses

SUBMITTER: Yang Z 

PROVIDER: S-EPMC11845889 | biostudies-literature | 2025 Feb

REPOSITORIES: biostudies-literature