Dataset Information

MRI restoration using edge-guided adversarial learning.

ABSTRACT: Magnetic resonance imaging (MRI) images acquired as multislice two-dimensional (2D) images present challenges when reformatted in orthogonal planes due to sparser sampling in the through-plane direction. Restoring the "missing" through-plane slices, or regions of an MRI image damaged by acquisition artifacts can be modeled as an image imputation task. In this work, we consider the damaged image data or missing through-plane slices as image masks and proposed an edge-guided generative adversarial network to restore brain MRI images. Inspired by the procedure of image inpainting, our proposed method decouples image repair into two stages: edge connection and contrast completion, both of which used general adversarial networks (GAN). We trained and tested on a dataset from the Human Connectome Project to test the application of our method for thick slice imputation, while we tested the artifact correction on clinical data and simulated datasets. Our Edge-Guided GAN had superior PSNR, SSIM, conspicuity and signal texture compared to traditional imputation tools, the Context Encoder and the Densely Connected Super Resolution Network with GAN (DCSRN-GAN). The proposed network may improve utilization of clinical 2D scans for 3D atlas generation and big-data comparative studies of brain morphometry.

SUBMITTER: Chai Y

PROVIDER: S-EPMC7977797 | biostudies-literature | 2020

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

MRI restoration using edge-guided adversarial learning.

Chai Yaqiong Y Xu Botian B Zhang Kangning K Lepore Natasha N Wood John J

IEEE access : practical innovations, open solutions 20200513

Magnetic resonance imaging (MRI) images acquired as multislice two-dimensional (2D) images present challenges when reformatted in orthogonal planes due to sparser sampling in the through-plane direction. Restoring the "missing" through-plane slices, or regions of an MRI image damaged by acquisition artifacts can be modeled as an image imputation task. In this work, we consider the damaged image data or missing through-plane slices as image masks and proposed an edge-guided generative adversarial ...[more]

PMID: 33747672

Similar Datasets

Project description:The aim of this study was to develop a deep neural network for respiratory motion compensation in free-breathing cine MRI and evaluate its performance. An adversarial autoencoder network was trained using unpaired training data from healthy volunteers and patients who underwent clinically indicated cardiac MRI examinations. A U-net structure was used for the encoder and decoder parts of the network and the code space was regularized by an adversarial objective. The autoencoder learns the identity map for the free-breathing motion-corrupted images and preserves the structural content of the images, while the discriminator, which interacts with the output of the encoder, forces the encoder to remove motion artifacts. The network was first evaluated based on data that were artificially corrupted with simulated rigid motion with regard to motion-correction accuracy and the presence of any artificially created structures. Subsequently, to demonstrate the feasibility of the proposed approach in vivo, our network was trained on respiratory motion-corrupted images in an unpaired manner and was tested on volunteer and patient data. In the simulation study, mean structural similarity index scores for the synthesized motion-corrupted images and motion-corrected images were 0.76 and 0.93 (out of 1), respectively. The proposed method increased the Tenengrad focus measure of the motion-corrupted images by 12% in the simulation study and by 7% in the in vivo study. The average overall subjective image quality scores for the motion-corrupted images, motion-corrected images and breath-held images were 2.5, 3.5 and 4.1 (out of 5.0), respectively. Nonparametric-paired comparisons showed that there was significant difference between the image quality scores of the motion-corrupted and breath-held images (P < .05); however, after correction there was no significant difference between the image quality scores of the motion-corrected and breath-held images. This feasibility study demonstrates the potential of an adversarial autoencoder network for correcting respiratory motion-related image artifacts without requiring paired data.

Project description:PurposeComplementary information obtained from multiple contrasts of tissue facilitates physicians assessing, diagnosing and planning treatment of a variety of diseases. However, acquiring multiple contrasts magnetic resonance images (MRI) for every patient using multiple pulse sequences is time-consuming and expensive, where, medical image synthesis has been demonstrated as an effective alternative. The purpose of this study is to develop a unified framework for multimodal MR image synthesis.MethodsA unified generative adversarial network consisting of only a single generator and a single discriminator was developed to learn the mappings among images of four different modalities. The generator took an image and its modality label as inputs and learned to synthesize the image in the target modality, while the discriminator was trained to distinguish between real and synthesized images and classify them to their corresponding modalities. The network was trained and tested using multimodal brain MRI consisting of four different contrasts which are T1-weighted (T1), T1-weighted and contrast-enhanced (T1c), T2-weighted (T2), and fluid-attenuated inversion recovery (Flair). Quantitative assessments of our proposed method were made through computing normalized mean absolute error (NMAE), peak signal-to-noise ratio (PSNR), structural similarity index measurement (SSIM), visual information fidelity (VIF), and naturalness image quality evaluator (NIQE).ResultsThe proposed model was trained and tested on a cohort of 274 glioma patients with well-aligned multi-types of MRI scans. After the model was trained, tests were conducted by using each of T1, T1c, T2, Flair as a single input modality to generate its respective rest modalities. Our proposed method shows high accuracy and robustness for image synthesis with arbitrary MRI modality that is available in the database as input. For example, with T1 as input modality, the NMAEs for the generated T1c, T2, Flair respectively are 0.034 ± 0.005, 0.041 ± 0.006, and 0.041 ± 0.006, the PSNRs respectively are 32.353 ± 2.525 dB, 30.016 ± 2.577 dB, and 29.091 ± 2.795 dB, the SSIMs are 0.974 ± 0.059, 0.969 ± 0.059, and 0.959 ± 0.059, the VIF are 0.750 ± 0.087, 0.706 ± 0.097, and 0.654 ± 0.062, and NIQE are 1.396 ± 0.401, 1.511 ± 0.460, and 1.259 ± 0.358, respectively.ConclusionsWe proposed a novel multimodal MR image synthesis method based on a unified generative adversarial network. The network takes an image and its modality label as inputs and synthesizes multimodal images in a single forward pass. The results demonstrate that the proposed method is able to accurately synthesize multimodal MR images from a single MR image.

Project description:Registration and fusion of magnetic resonance imaging (MRI) and transrectal ultrasound (TRUS) of the prostate can provide guidance for prostate brachytherapy. However, accurate registration remains a challenging task due to the lack of ground truth regarding voxel-level spatial correspondence, limited field of view, low contrast-to-noise ratio, and signal-to-noise ratio in TRUS. In this study, we proposed a fully automated deep learning approach based on a weakly supervised method to address these issues. We employed deep learning techniques to combine image segmentation and registration, including affine and nonrigid registration, to perform an automated deformable MRI-TRUS registration. To start with, we trained two separate fully convolutional neural networks (CNNs) to perform a pixel-wise prediction for MRI and TRUS prostate segmentation. Then, to provide the initialization of the registration, a 2D CNN was used to register MRI-TRUS prostate images using an affine registration. After that, a 3D UNET-like network was applied for nonrigid registration. For both the affine and nonrigid registration, pairs of MRI-TRUS labels were concatenated and fed into the neural networks for training. Due to the unavailability of ground-truth voxel-level correspondences and the lack of accurate intensity-based image similarity measures, we propose to use prostate label-derived volume overlaps and surface agreements as an optimization objective function for weakly supervised network training. Specifically, we proposed a hybrid loss function that integrated a Dice loss, a surface-based loss, and a bending energy regularization loss for the nonrigid registration. The Dice and surface-based losses were used to encourage the alignment of the prostate label between the MRI and the TRUS. The bending energy regularization loss was used to achieve a smooth deformation field. Thirty-six sets of patient data were used to test our registration method. The image registration results showed that the deformed MR image aligned well with the TRUS image, as judged by corresponding cysts and calcifications in the prostate. The quantitative results showed that our method produced a mean target registration error (TRE) of 2.53 ± 1.39 mm and a mean Dice loss of 0.91 ± 0.02. The mean surface distance (MSD) and Hausdorff distance (HD) between the registered MR prostate shape and TRUS prostate shape were 0.88 and 4.41 mm, respectively. This work presents a deep learning-based, weakly supervised network for accurate MRI-TRUS image registration. Our proposed method has achieved promising registration performance in terms of Dice loss, TRE, MSD, and HD.

Project description:PurposeMRI-guided adaptive radiation therapy (MRgART), particularly daily online adaptive replanning (OLAR) can substantially improve radiation therapy delivery, however, it can be labor-intensive and time-consuming. Currently, the decision to perform OLAR for a treatment fraction is determined subjectively. In this work, we develop a machine learning algorithm based on structural similarity index measure (SSIM) and change in entropy to quickly and objectively determine whether OLAR is necessary for a daily MRI set.MethodsA total of 109 daily MRI sets acquired on a 1.5T MR-Linac during MRgART for 22 pancreatic cancer patients each treated with five fractions were retrospectively analyzed. For each daily MRI set, OLAR and reposition (No-OLAR) plans were created and the superior plan with the daily fraction determined per clinical dose-volume criteria. SSIM and entropy maps were extracted from each daily MRI set, with respect to its reference (e.g., dry-run) MRI in the region enclosed by 50-100% isodose surfaces. A total of six common features were extracted from SSIM maps. Pearson's rank correlation coefficient was utilized to rule out redundant SSIM features. A t-test was used to determine significant SSIM features which were combined with the change in entropy to develop anensemble machine classifier with fivefold cross validation. The performance of the classifier was evaluated using the area under the curve (AUC) of the receiver operating characteristic curve.ResultsA machine learning classifier model using two SSIM features (mean and full width at half maximum) and change in entropy was determined to be able to significantly discriminate between No-OLAR and OLAR groups. The obtained machine learning ensemble classifier can predict OLAR necessity with a cross validated AUC of 0.93. Misclassification was found primarily for No-OLAR cases with dosimetric plan quality closely comparable to the corresponding OLAR plans, thus, are not a major practical concern.ConclusionA machine learning classifier based on simple first-order image features, that is, SSIM features and change in entropy, was developed to determine when OLAR is necessary for a daily MRI set with practical acceptable prediction accuracy. This classifier may be implemented in the MRgART process to automatically and objectively determine if OLAR is required following daily MRI.

Dataset Information

MRI restoration using edge-guided adversarial learning.

Publications

MRI restoration using edge-guided adversarial learning.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets