The simulation of judgment in LLMs.

ABSTRACT: Large Language Models (LLMs) are increasingly embedded in evaluative processes, from information filtering to assessing and addressing knowledge gaps through explanation and credibility judgments. This raises the need to examine how such evaluations are built, what assumptions they rely on, and how their strategies diverge from those of humans. We benchmark six LLMs against expert ratings-NewsGuard and Media Bias/Fact Check-and against human judgments collected through a controlled experiment. We use news domains purely as a controlled benchmark for evaluative tasks, focusing on the underlying mechanisms rather than on news classification per se. To enable direct comparison, we implement a structured agentic framework in which both models and nonexpert participants follow the same evaluati

SUBMITTER: Loru E 

PROVIDER: S-EPMC12557803 | biostudies-literature | 2025 Oct

REPOSITORIES: biostudies-literature