Advancing surgical VQA with scene graph knowledge.

ABSTRACT:

Purpose

The modern operating room is becoming increasingly complex, requiring innovative intra-operative support systems. While the focus of surgical data science has largely been on video analysis, integrating surgical computer vision with natural language capabilities is emerging as a necessity. Our work aims to advance visual question answering (VQA) in the surgical context with scene graph knowledge, addressing two main challenges in the current surgical VQA systems: removing question-condition bias in the surgical VQA dataset and incorporating scene-aware reasoning in the surgical VQA model design.

Methods

First, we propose a surgical scene graph-based dataset, SSG-VQA, generated by employing segmentation and detection models on publicly available datasets. We build sur

SUBMITTER: Yuan K 

PROVIDER: S-EPMC11231006 | biostudies-literature | 2024 Jul

REPOSITORIES: biostudies-literature