Hallucination, monofacts, and miscalibration: An empirical investigation.

ABSTRACT: Hallucinated facts in large language models have recently been shown to obey a statistical lower bound determined by the monofact rate (related to the classical Good-Turing missing mass estimator) minus model miscalibration [A. T. Kalai, S. S. Vempala, "Calibrated language models must hallucinate" in Proceedings of the 56th Annual ACM Symposium on Theory of Computing (STOC) (New York, NY, USA, 2024), pp. 160-171]. We present empirical investigation of this three-way relationship in classical [Formula: see text]-gram models and fine-tuned transformer models. By generating training data from Pareto distributions with varying shape parameters, we systematically control the monofact rate and establish its positive relationship with hallucination. To bridge theory and practice, we derive

SUBMITTER: Miao MM 

PROVIDER: S-EPMC12933039 | biostudies-literature | 2026 Feb

REPOSITORIES: biostudies-literature