Benchmarking large multimodal models for ophthalmic visual question answering with OphthalWeChat.

ABSTRACT:

Purpose

To develop a bilingual multimodal visual question answering (VQA) benchmark for evaluating Vision-language models (VLMs) in ophthalmology.

Methods

In this cross-sectional study, ophthalmic image posts and associated captions published between Jan 1, 2016, and Dec 31, 2024, were collected from WeChat Official Accounts. Based on these captions, bilingual question-answer (QA) pairs in Chinese and English were generated using GPT-4o-mini. QA pairs were categorized into six subsets by question type and language: binary (Binary_CN, Binary_EN), single-choice (Single-choice_CN, Single-choice_EN), and open-ended (Open-ended_CN, Open-ended_EN). The benchmark was used to evaluate six VLMs: GPT-4o, Gemini 2.0 Flash, Qwen2.5-VL-72B-Instruct, Janus-Pro-7B, InternVL3-8B, and Health

SUBMITTER: Xu P 

PROVIDER: S-EPMC12834899 | biostudies-literature | 2026 Feb-Mar

REPOSITORIES: biostudies-literature