Dataset Information

Representation Disentanglement for Multi-modal Brain MRI Analysis.

ABSTRACT: Multi-modal MRIs are widely used in neuroimaging applications since different MR sequences provide complementary information about brain structures. Recent works have suggested that multi-modal deep learning analysis can benefit from explicitly disentangling anatomical (shape) and modality (appearance) information into separate image presentations. In this work, we challenge mainstream strategies by showing that they do not naturally lead to representation disentanglement both in theory and in practice. To address this issue, we propose a margin loss that regularizes the similarity in relationships of the representations across subjects and modalities. To enable robust training, we further use a conditional convolution to design a single model for encoding images of all modalities. Lastly, we propose a fusion function to combine the disentangled anatomical representations as a set of modality-invariant features for downstream tasks. We evaluate the proposed method on three multi-modal neuroimaging datasets. Experiments show that our proposed method can achieve superior disentangled representations compared to existing disentanglement strategies. Results also indicate that the fused anatomical representation has potential in the downstream task of zero-dose PET reconstruction and brain tumor segmentation.

SUBMITTER: Ouyang J

PROVIDER: S-EPMC8844656 | biostudies-literature | 2021 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Representation Disentanglement for Multi-modal Brain MRI Analysis.

Ouyang Jiahong J Adeli Ehsan E Pohl Kilian M KM Zhao Qingyu Q Zaharchuk Greg G

Information processing in medical imaging : proceedings of the ... conference 20210614

Multi-modal MRIs are widely used in neuroimaging applications since different MR sequences provide complementary information about brain structures. Recent works have suggested that multi-modal deep learning analysis can benefit from explicitly disentangling anatomical (shape) and modality (appearance) information into separate image presentations. In this work, we challenge mainstream strategies by showing that they do not naturally lead to representation disentanglement both in theory and in p ...[more]

PMID: 35173402

Similar Datasets

Project description:Idiopathic rapid eye movement sleep behavior disorder (iRBD) is a condition that often evolves into Parkinson's disease (PD). Therefore, by monitoring iRBD it is possible to track the neurodegeneration of individuals who may progress to PD. Here we aimed at piloting the characterization of brain tissue properties in mid-brain subcortical regions of 10 healthy subjects, 8 iRBD, and 9 early-diagnosed PD. We used a battery of magnetic resonance imaging (MRI) contrasts at 3 T, including adiabatic and non-adiabatic rotating frame techniques developed by our group, along with diffusion tensor imaging (DTI) and resting-state fMRI. Adiabatic T1ρ and T2ρ, and non-adiabatic RAFF4 (Relaxation Along a Fictitious Field in the rotating frame of rank 4) were found to have lower coefficient of variations and higher sensitivity to detect group differences as compared to DTI parameters such as fractional anisotropy and mean diffusivity. Significantly longer T1ρ were observed in the amygdala of PD subjects vs. controls, along with a trend of lower functional connectivity as measured by regional homogeneity, thereby supporting the notion that amygdalar dysfunction occurs in PD. Significant abnormalities in reward networks occurred in iRBD subjects, who manifested lower network strength of the accumbens. In agreement with previous studies, significantly longer T1ρ occurred in the substantia nigra compacta of PD vs. controls, indicative of neuronal degeneration, while regional homogeneity was lower in the substantia nigra reticulata. Finally, other trend-level findings were observed, i.e., lower RAFF4 and T2ρ in the midbrain of iRBD subjects vs. controls, possibly indicating changes in non-motor features as opposed to motor function in the iRBD group. We conclude that rotating frame relaxation methods along with functional connectivity measures are valuable to characterize iRBD and PD subjects, and with proper validation in larger cohorts may provide pathological signatures of iRBD and PD.

Project description:BackgroundPseudo-computed tomography (pCT) quality is a crucial issue in magnetic resonance image (MRI)-only brain stereotactic radiotherapy (SRT), so this study systematically evaluated it from the multi-modal radiomics perspective.Methods34 cases (< 30 cm³) were retrospectively included (2021.9-2022.10). For each case, both CT and MRI scans were performed at simulation, and pCT was generated by a convolutional neural network (CNN) from planning MRI. Conformal arc or volumetric modulated arc technique was used to optimize the dose distribution. The SRT dose was compared between pCT and planning CT with dose volume histogram (DVH) metrics and gamma index. Wilcoxon test and Spearman analysis were used to identify key factors associated with dose deviations. Additionally, original image features were extracted for radiomic analysis. Tumor control probability (TCP) and normal tissue complication probability (NTCP) were employed for efficacy evaluation.ResultsThere was no significant difference between pCT and planning CT except for radiomics. The mean value of Hounsfield unit of the planning CT was slightly higher than that of pCT. The Gadolinium-based agents in planning MRI could increase DVH metrics deviation slightly. The median local gamma passing rates (1%/1 mm) between planning CTs and pCTs (non-contrast) was 92.6% (range 63.5-99.6%). Also, differences were observed in more than 85% of original radiomic features. The mean absolute deviation in TCP was 0.03%, and the NTCP difference was below 0.02%, except for the normal brain, which had a 0.16% difference. In addition, the number of SRT fractions and lesions, and lesion morphology could influence dose deviation.ConclusionsThis is the first multi-modal radiomics analysis of CNN-based pCT from planning MRI for SRT of small brain lesions, covering dosiomics and radiomics. The findings suggest the potential of pCT in SRT plan design and efficacy prediction, but caution needs to be taken for radiomic analysis.

Dataset Information

Representation Disentanglement for Multi-modal Brain MRI Analysis.

Publications

Representation Disentanglement for Multi-modal Brain MRI Analysis.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets