Performance of large language models ChatGPT and Gemini in child and adolescent psychiatry knowledge assessment.

ABSTRACT:

Objective

This study evaluates the performance of four large language models-ChatGPT 4o, ChatGPT o1-mini, Gemini 2.0 Flash, and Gemini 1.5 Flash-in answering multiple-choice questions in child and adolescent psychiatry to assess their level of factual knowledge in the field.

Methods

A total of 150 standardized multiple-choice questions from a specialty board review study guide were selected, ensuring a representative distribution across different topics. Each question had five possible answers, with only one correct option. To account for the stochastic nature of large language models, each question was asked 10 times with randomized answer orders to minimize known biases. Accuracy for each question was assessed as the percentage of correct answers across 10 requests. We cal

SUBMITTER: Neubauer JC 

PROVIDER: S-EPMC12449005 | biostudies-literature | 2025

REPOSITORIES: biostudies-literature