Dataset Information

Deep hashing for global registration of untracked 2D laparoscopic ultrasound to CT.

ABSTRACT:

Purpose

The registration of Laparoscopic Ultrasound (LUS) to CT can enhance the safety of laparoscopic liver surgery by providing the surgeon with awareness on the relative positioning between critical vessels and a tumour. In an effort to provide a translatable solution for this poorly constrained problem, Content-based Image Retrieval (CBIR) based on vessel information has been suggested as a method for obtaining a global coarse registration without using tracking information. However, the performance of these frameworks is limited by the use of non-generalisable handcrafted vessel features.

Methods

We propose the use of a Deep Hashing (DH) network to directly convert vessel images from both LUS and CT into fixed size hash codes. During training, these codes are learnt from a patient-specific CT scan by supplying the network with triplets of vessel images which include both a registered and a mis-registered pair. Once hash codes have been learnt, they can be used to perform registration with CBIR methods.

Results

We test a CBIR pipeline on 11 sequences of untracked LUS distributed across 5 clinical cases. Compared to a handcrafted feature approach, our model improves the registration success rate significantly from 48% to 61%, considering a 20 mm error as the threshold for a successful coarse registration.

Conclusions

We present the first DH framework for interventional multi-modal registration tasks. The presented approach is easily generalisable to other registration problems, does not require annotated data for training, and may promote the translation of these techniques.

SUBMITTER: Ramalhinho J

PROVIDER: S-EPMC9307559 | biostudies-literature | 2022 Aug

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Deep hashing for global registration of untracked 2D laparoscopic ultrasound to CT.

Ramalhinho João J Koo Bongjin B Montaña-Brown Nina N Saeed Shaheer U SU Bonmati Ester E Gurusamy Kurinchi K Pereira Stephen P SP Davidson Brian B Hu Yipeng Y Clarkson Matthew J MJ

International journal of computer assisted radiology and surgery 20220402 8

<h4>Purpose</h4>The registration of Laparoscopic Ultrasound (LUS) to CT can enhance the safety of laparoscopic liver surgery by providing the surgeon with awareness on the relative positioning between critical vessels and a tumour. In an effort to provide a translatable solution for this poorly constrained problem, Content-based Image Retrieval (CBIR) based on vessel information has been suggested as a method for obtaining a global coarse registration without using tracking information. However, ...[more]

PMID: 35366130

Similar Datasets

Project description:In prostate radiotherapy, setup errors with respect to the patient's bony anatomy can be reduced by aligning 2D megavoltage (MV) portal images acquired during treatment to a reference 3D kilovoltage (kV) CT acquired for treatment planning purposes. The purpose of this study was to evaluate a fully automated 2D-3D registration algorithm to quantify setup errors in 3D through the alignment of line-enhanced portal images and digitally reconstructed radiographs computed from the CT. The line-enhanced images were obtained by correlating the images with a filter bank of short line segments, or "sticks" at different orientations. The proposed methods were validated on (1) accurately collected gold-standard data consisting of a 3D kV cone-beam CT scan of an anthropomorphic phantom of the pelvis and 2D MV portal images in the anterior-posterior (AP) view acquired at 15 different poses and (2) a conventional 3D kV CT scan and weekly 2D MV AP portal images of a patient over 8 weeks. The mean (and standard deviation) of the absolute registration error for rotations around the right-lateral (RL), inferior-superior (IS), and posterior-anterior (PA) axes were 0.212 degree (0.214 degree), 0.055 degree (0.033 degree) and 0.041 degree (0.039 degree), respectively. The corresponding registration errors for translations along the RL, IS, and PA axes were 0.161 (0.131) mm, 0.096 (0.033) mm, and 0.612 (0.485) mm. The mean (and standard deviation) of the total registration error was 0.778 (0.543) mm. Registration on the patient images was successful in all eight cases as determined visually. The results indicate that it is feasible to automatically enhance features in MV portal images of the pelvis for use within a completely automated 2D-3D registration framework for the accurate determination of patient setup errors. They also indicate that it is feasible to estimate all six transformation parameters from a 3D CT of the pelvis and a single portal image in the AP view.

Project description:Background and purposeTranscranial MR imaging-guided focused ultrasound is a promising novel technique to treat multiple disorders and diseases. Planning for transcranial MR imaging-guided focused ultrasound requires both a CT scan for skull density estimation and treatment-planning simulation and an MR imaging for target identification. It is desirable to simplify the clinical workflow of transcranial MR imaging-guided focused ultrasound treatment planning. The purpose of this study was to examine the feasibility of deep learning techniques to convert MR imaging ultrashort TE images directly to synthetic CT of the skull images for use in transcranial MR imaging-guided focused ultrasound treatment planning.Materials and methodsThe U-Net neural network was trained and tested on data obtained from 41 subjects (mean age, 66.4 ± 11.0 years; 15 women). The derived neural network model was evaluated using a k-fold cross-validation method. Derived acoustic properties were verified by comparing the whole skull-density ratio from deep learning synthesized CT of the skull with the reference CT of the skull. In addition, acoustic and temperature simulations were performed using the deep learning CT to predict the target temperature rise during transcranial MR imaging-guided focused ultrasound.ResultsThe derived deep learning model generates synthetic CT of the skull images that are highly comparable with the true CT of the skull images. Their intensities in Hounsfield units have a spatial correlation coefficient of 0.80 ± 0.08, a mean absolute error of 104.57 ± 21.33 HU, and a subject-wise correlation coefficient of 0.91. Furthermore, deep learning CT of the skull is reliable in the skull-density ratio estimation (r = 0.96). A simulation study showed that both the peak target temperatures and temperature distribution from deep learning CT are comparable with those of the reference CT.ConclusionsThe deep learning method can be used to simplify workflow associated with transcranial MR imaging-guided focused ultrasound.

Project description:BackgroundAdaptive radiation treatment (ART) for locally advanced pancreatic cancer (LAPC) requires consistently accurate segmentation of the extremely mobile gastrointestinal (GI) organs at risk (OAR) including the stomach, duodenum, large and small bowel. Also, due to lack of sufficiently accurate and fast deformable image registration (DIR), accumulated dose to the GI OARs is currently only approximated, further limiting the ability to more precisely adapt treatments.PurposeDevelop a 3-D Progressively refined joint Registration-Segmentation (ProRSeg) deep network to deformably align and segment treatment fraction magnetic resonance images (MRI)s, then evaluate segmentation accuracy, registration consistency, and feasibility for OAR dose accumulation.MethodProRSeg was trained using five-fold cross-validation with 110 T2-weighted MRI acquired at five treatment fractions from 10 different patients, taking care that same patient scans were not placed in training and testing folds. Segmentation accuracy was measured using Dice similarity coefficient (DSC) and Hausdorff distance at 95th percentile (HD95). Registration consistency was measured using coefficient of variation (CV) in displacement of OARs. Statistical comparison to other deep learning and iterative registration methods were done using the Kruskal-Wallis test, followed by pair-wise comparisons with Bonferroni correction applied for multiple testing. Ablation tests and accuracy comparisons against multiple methods were done. Finally, applicability of ProRSeg to segment cone-beam CT (CBCT) scans was evaluated on a publicly available dataset of 80 scans using five-fold cross-validation.ResultsProRSeg processed 3D volumes (128 × 192 × 128) in 3 s on a NVIDIA Tesla V100 GPU. It's segmentations were significantly more accurate ( p<0.001$p<0.001$ ) than compared methods, achieving a DSC of 0.94 ±0.02 for liver, 0.88±0.04 for large bowel, 0.78±0.03 for small bowel and 0.82±0.04 for stomach-duodenum from MRI. ProRSeg achieved a DSC of 0.72±0.01 for small bowel and 0.76±0.03 for stomach-duodenum from public CBCT dataset. ProRSeg registrations resulted in the lowest CV in displacement (stomach-duodenum CVx$CV_{x}$ : 0.75%, CVy$CV_{y}$ : 0.73%, and CVz$CV_{z}$ : 0.81%; small bowel CVx$CV_{x}$ : 0.80%, CVy$CV_{y}$ : 0.80%, and CVz$CV_{z}$ : 0.68%; large bowel CVx$CV_{x}$ : 0.71%, CVy$CV_{y}$ : 0.81%, and CVz$CV_{z}$ : 0.75%). ProRSeg based dose accumulation accounting for intra-fraction (pre-treatment to post-treatment MRI scan) and inter-fraction motion showed that the organ dose constraints were violated in four patients for stomach-duodenum and for three patients for small bowel. Study limitations include lack of independent testing and ground truth phantom datasets to measure dose accumulation accuracy.ConclusionsProRSeg produced more accurate and consistent GI OARs segmentation and DIR of MRI and CBCTs compared to multiple methods. Preliminary results indicates feasibility for OAR dose accumulation using ProRSeg.

Dataset Information

Deep hashing for global registration of untracked 2D laparoscopic ultrasound to CT.

Purpose

Methods

Results

Conclusions

Publications

Deep hashing for global registration of untracked 2D laparoscopic ultrasound to CT.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets