Benchmarking retrieval-augmented large language models in biomedical NLP: Application, robustness, and self-awareness.

ABSTRACT: To reduce hallucinations in large language models (LLMs), retrieval-augmented LLMs (RALs) retrieve supporting knowledge from external databases. However, their performance on biomedical natural language processing (NLP) tasks remains underexplored. We introduce Biomedical Retrieval-Augmented Generation Benchmark, a comprehensive evaluation framework assessing RALs across five biomedical NLP tasks and 11 datasets, using four testbeds: unlabeled robustness, counterfactual robustness, diverse robustness, and self-awareness. To improve RALs' robustness and negative awareness, we propose a detect-and-correct strategy and a contrastive learning approach. Experimental results show that RALs generally outperform standard LLMs on most biomedical tasks, but still struggle with robustness and self-aw

SUBMITTER: Li M 

PROVIDER: S-EPMC12637297 | biostudies-literature | 2025 Nov

REPOSITORIES: biostudies-literature