Development of a large-scale medical visual question-answering dataset.

ABSTRACT:

Background

Medical Visual Question Answering (MedVQA) enhances diagnostic accuracy and healthcare delivery by leveraging artificial intelligence to interpret medical images. This study aims to redefine MedVQA as a generation task that mirrors human-machine interaction and to develop a model capable of integrating complex visual and textual information.

Methods

We constructed a large-scale medical visual-question answering dataset, PMC-VQA, containing 227,000 VQA pairs across 149,000 images that span various modalities and diseases. We introduced a generative model that aligns visual information from a pre-trained vision encoder with a large language model. This model was initially trained on PMC-VQA and subsequently fine-tuned on multiple public benchmarks.

Results

He

SUBMITTER: Zhang X 

PROVIDER: S-EPMC11663219 | biostudies-literature | 2024 Dec

REPOSITORIES: biostudies-literature