Dataset Information

Capsule robot pose and mechanism state detection in ultrasound using attention-based hierarchical deep learning.

ABSTRACT: Ingestible robotic capsules with locomotion capabilities and on-board sampling mechanism have great potential for non-invasive diagnostic and interventional use in the gastrointestinal tract. Real-time tracking of capsule location and operational state is necessary for clinical application, yet remains a significant challenge. To this end, we propose an approach that can simultaneously determine the mechanism state and in-plane 2D pose of millimeter capsule robots in an anatomically representative environment using ultrasound imaging. Our work proposes an attention-based hierarchical deep learning approach and adapts the success of transfer learning towards solving the multi-task tracking problem with limited dataset. To train the neural networks, we generate a representative dataset of a robotic capsule within ex-vivo porcine stomachs. Experimental results show that the accuracy of capsule state classification is 97%, and the mean estimation errors for orientation and centroid position are 2.0 degrees and 0.24 mm (1.7% of the capsule's body length) on the hold-out test set. Accurate detection of the capsule while manipulated by an external magnet in a porcine stomach and colon is also demonstrated. The results suggest our proposed method has the potential for advancing the wireless capsule-based technologies by providing accurate detection of capsule robots in clinical scenarios.

SUBMITTER: Liu X

PROVIDER: S-EPMC9729303 | biostudies-literature | 2022 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Capsule robot pose and mechanism state detection in ultrasound using attention-based hierarchical deep learning.

Liu Xiaoyun X Esser Daniel D Wagstaff Brandon B Zavodni Anna A Matsuura Naomi N Kelly Jonathan J Diller Eric E

Scientific reports 20221207 1

Ingestible robotic capsules with locomotion capabilities and on-board sampling mechanism have great potential for non-invasive diagnostic and interventional use in the gastrointestinal tract. Real-time tracking of capsule location and operational state is necessary for clinical application, yet remains a significant challenge. To this end, we propose an approach that can simultaneously determine the mechanism state and in-plane 2D pose of millimeter capsule robots in an anatomically representati ...[more]

PMID: 36476715

Similar Datasets

Project description:Lameness affects animal mobility, causing pain and discomfort. Lameness in early stages often goes undetected due to a lack of observation, precision, and reliability. Automated and non-invasive systems offer precision and detection ease and may improve animal welfare. This study was conducted to create a repository of images and videos of sows with different locomotion scores. Our goal is to develop a computer vision model for automatically identifying specific points on the sow's body. The automatic identification and ability to track specific body areas, will allow us to conduct kinematic studies with the aim of facilitating the detection of lameness using deep learning. The video database was collected on a pig farm with a scenario built to allow filming of sows in locomotion with different lameness scores. Two stereo cameras were used to record 2D videos images. Thirteen locomotion experts assessed the videos using the Locomotion Score System developed by Zinpro Corporation. From this annotated repository, computational models were trained and tested using the open-source deep learning-based animal pose tracking framework SLEAP (Social LEAP Estimates Animal Poses). The top-performing models were constructed using the LEAP architecture to accurately track 6 (lateral view) and 10 (dorsal view) skeleton keypoints. The architecture achieved average precisions values of 0.90 and 0.72, average distances of 6.83 and 11.37 in pixel, and similarities of 0.94 and 0.86 for the lateral and dorsal views, respectively. These computational models are proposed as a Precision Livestock Farming tool and method for identifying and estimating postures in pigs automatically and objectively. The 2D video image repository with different pig locomotion scores can be used as a tool for teaching and research. Based on our skeleton keypoint classification results, an automatic system could be developed. This could contribute to the objective assessment of locomotion scores in sows, improving their welfare.

Project description:BackgroundAccurate diagnosis of unexplained cervical lymphadenopathy (CLA) using medical images heavily relies on the experience of radiologists, which is even worse for CLA patients in underdeveloped countries and regions, because of lack of expertise and reliable medical history. This study aimed to develop a deep learning (DL) radiomics model based on B-mode and color Doppler ultrasound images for assisting radiologists to improve their diagnoses of the etiology of unexplained CLA.MethodsPatients with unexplained CLA who received ultrasound examinations from three hospitals located in underdeveloped areas of China were retrospectively enrolled. They were all pathologically confirmed with reactive hyperplasia, tuberculous lymphadenitis, lymphoma, or metastatic carcinoma. By mimicking the diagnosis logic of radiologists, three DL sub-models were developed to achieve the primary diagnosis of benign and malignant, the secondary diagnosis of reactive hyperplasia and tuberculous lymphadenitis in benign candidates, and of lymphoma and metastatic carcinoma in malignant candidates, respectively. Then, a CLA hierarchical diagnostic model (CLA-HDM) integrating all sub-models was proposed to classify the specific etiology of each unexplained CLA. The assistant effectiveness of CLA-HDM was assessed by comparing six radiologists between without and with using the DL-based classification and heatmap guidance.ResultsA total of 763 patients with unexplained CLA were enrolled and were split into the training cohort (n=395), internal testing cohort (n=171), and external testing cohorts 1 (n=105) and 2 (n=92). The CLA-HDM for diagnosing four common etiologies of unexplained CLA achieved AUCs of 0.873 (95% CI: 0.838-0.908), 0.837 (95% CI: 0.789-0.889), and 0.840 (95% CI: 0.789-0.898) in the three testing cohorts, respectively, which was systematically more accurate than all the participating radiologists. With its assistance, the accuracy, sensitivity, and specificity of six radiologists with different levels of experience were generally improved, reducing the false-negative rate of 2.2-10% and the false-positive rate of 0.7-3.1%.ConclusionsMulti-cohort testing demonstrated our DL model integrating dual-modality ultrasound images achieved accurate diagnosis of unexplained CLA. With its assistance, the gap between radiologists with different levels of experience was narrowed, which is potentially of great significance for benefiting CLA patients in underdeveloped countries and regions worldwide.

Dataset Information

Capsule robot pose and mechanism state detection in ultrasound using attention-based hierarchical deep learning.

Publications

Capsule robot pose and mechanism state detection in ultrasound using attention-based hierarchical deep learning.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets