There are significant differences among artificial intelligence large language models when answering scientific questions.

ABSTRACT:

Introduction

This study investigates the efficacy of large language models (LLMs) for generating accurate scientific responses through a comparative evaluation of five prominent free models: Claude 3.5 Sonnet, Gemini, ChatGPT 4o, Mistral Large 2, and Llama 3.1 70B.

Methods

Sixteen expert scientific reviewers assessed these models in terms of depth, accuracy, relevance, and clarity.

Results

Claude 3.5 Sonnet emerged as the highest scoring model, followed by Gemini, with notable variability among the other models. Additionally, retrieval-augmented generation (RAG) techniques were applied to improve LLM performance, and prompts were refined to improve answers. The results indicate that although LLMs such as Claude 3.5 Sonnet have potential for scientific tasks, other mod

SUBMITTER: Alvarez-Martinez FJ 

PROVIDER: S-EPMC12547693 | biostudies-literature | 2025

REPOSITORIES: biostudies-literature