Large language models underperform in European general surgery board examinations: a comparative study with experts and surgical residents.

ABSTRACT: Artificial intelligence (AI) has become a transformative tool in medical education and assessment. Despite advancements, AI models such as GPT-4o demonstrate variable performance on high-stakes examinations. This study compared the performance of four AI models (Llama-3, Gemini, GPT-4o, and Copilot) with specialists and residents on European General Surgery Board test questions, focusing on accuracy across question formats, lengths, and difficulty levels. A total of 120 multiple-choice questions were systematically sampled from the General Surgery Examination and Board Review question bank using a structured randomization protocol. The questions were administered via Google Forms to four large language models (Llama-3, GPT-4o, Gemini, and Copilot) and 30 surgeons (15 board-certified specia

SUBMITTER: Gul MC 

PROVIDER: S-EPMC12374362 | biostudies-literature | 2025 Aug

REPOSITORIES: biostudies-literature