Capsule robot pose and mechanism state detection in ultrasound using attention-based hierarchical deep learning.

ABSTRACT: Ingestible robotic capsules with locomotion capabilities and on-board sampling mechanism have great potential for non-invasive diagnostic and interventional use in the gastrointestinal tract. Real-time tracking of capsule location and operational state is necessary for clinical application, yet remains a significant challenge. To this end, we propose an approach that can simultaneously determine the mechanism state and in-plane 2D pose of millimeter capsule robots in an anatomically representative environment using ultrasound imaging. Our work proposes an attention-based hierarchical deep learning approach and adapts the success of transfer learning towards solving the multi-task tracking problem with limited dataset. To train the neural networks, we generate a representative dataset of a

SUBMITTER: Liu X 

PROVIDER: S-EPMC9729303 | biostudies-literature | 2022 Dec

REPOSITORIES: biostudies-literature