SDA-CLIP: surgical visual domain adaptation using video and text labels.

ABSTRACT:

Background

Surgical action recognition is an essential technology in context-aware-based autonomous surgery, whereas the accuracy is limited by clinical dataset scale. Leveraging surgical videos from virtual reality (VR) simulations to research algorithms for the clinical domain application, also known as domain adaptation, can effectively reduce the cost of data acquisition and annotation, and protect patient privacy.

Methods

We introduced a surgical domain adaptation method based on the contrastive language-image pretraining model (SDA-CLIP) to recognize cross-domain surgical action. Specifically, we utilized the Vision Transformer (ViT) and Transformer to extract video and text embeddings, respectively. Text embedding was developed as a bridge between VR and clinical doma

SUBMITTER: Li Y 

PROVIDER: S-EPMC10585553 | biostudies-literature | 2023 Oct

REPOSITORIES: biostudies-literature