Prompt-dependent performance of multimodal AI model in oral diagnosis: a comprehensive analysis of accuracy, narrative quality, calibration, and latency versus human experts.

ABSTRACT: Prompt design is a critical yet underexplored factor influencing the diagnostic performance of large language models (LLMs). Gemini Pro 2.5 shows promise in multimodal reasoning, but no prior study has systematically compared prompt structures in oral datasets against expert benchmarks. This study aimed to evaluate the diagnostic performance of a multimodal LLM (Gemini Pro 2.5) under different prompting strategies compared with oral medicine experts using prospective, histopathology-verified clinical vignettes. In a prospective, paired diagnostic accuracy study, Gemini pro 2.5 (a multimodal LLM) was evaluated under three prompting strategies: Direct (P-1), Chain-of-Thought (P-2), and Self-Reflection (P-3) on 300 oral lesion cases with histopathologic confirmation. Each prompt was applied t

SUBMITTER: Hassanein FEA 

PROVIDER: S-EPMC12575769 | biostudies-literature | 2025 Oct

REPOSITORIES: biostudies-literature