Deep Multiview Learning to Identify Population Structure with Multimodal Imaging.
Ontology highlight
ABSTRACT:
SUBMITTER: Feng Y
PROVIDER: S-EPMC7917002 | biostudies-literature | 2020 Oct
REPOSITORIES: biostudies-literature
Publications
Deep Multiview Learning to Identify Population Structure with Multimodal Imaging.
Feng Yixue Y Liu Kefei K Kim Mansu M Long Qi Q Yao Xiaohui X Shen Li L
Proceedings. IEEE International Symposium on Bioinformatics and Bioengineering 20201001
We present an effective deep multiview learning framework to identify population structure using multimodal imaging data. Our approach is based on canonical correlation analysis (CCA). We propose to use deep generalized CCA (DGCCA) to learn a shared latent representation of non-linearly mapped and maximally correlated components from multiple imaging modalities with reduced dimensionality. In our empirical study, this representation is shown to effectively capture more variance in original data ...[more]
Similar Datasets
Project description:BackgroundIn Alzheimer's Diseases (AD) research, multimodal imaging analysis can unveil complementary information from multiple imaging modalities and further our understanding of the disease. One application is to discover disease subtypes using unsupervised clustering. However, existing clustering methods are often applied to input features directly, and could suffer from the curse of dimensionality with high-dimensional multimodal data. The purpose of our study is to identify multimodal imaging-driven subtypes in Mild Cognitive Impairment (MCI) participants using a multiview learning framework based on Deep Generalized Canonical Correlation Analysis (DGCCA), to learn shared latent representation with low dimensions from 3 neuroimaging modalities.ResultsDGCCA applies non-linear transformation to input views using neural networks and is able to learn correlated embeddings with low dimensions that capture more variance than its linear counterpart, generalized CCA (GCCA). We designed experiments to compare DGCCA embeddings with single modality features and GCCA embeddings by generating 2 subtypes from each feature set using unsupervised clustering. In our validation studies, we found that amyloid PET imaging has the most discriminative features compared with structural MRI and FDG PET which DGCCA learns from but not GCCA. DGCCA subtypes show differential measures in 5 cognitive assessments, 6 brain volume measures, and conversion to AD patterns. In addition, DGCCA MCI subtypes confirmed AD genetic markers with strong signals that existing late MCI group did not identify.ConclusionOverall, DGCCA is able to learn effective low dimensional embeddings from multimodal data by learning non-linear projections. MCI subtypes generated from DGCCA embeddings are different from existing early and late MCI groups and show most similarity with those identified by amyloid PET features. In our validation studies, DGCCA subtypes show distinct patterns in cognitive measures, brain volumes, and are able to identify AD genetic markers. These findings indicate the promise of the imaging-driven subtypes and their power in revealing disease structures beyond early and late stage MCI.
| S-EPMC9523890 | biostudies-literature
Project description:BackgroundTechnological advances have enabled the generation of unique and complementary types of data or views (e.g. genomics, proteomics, metabolomics) and opened up a new era in multiview learning research with the potential to lead to new biomedical discoveries.ResultsWe propose iDeepViewLearn (Interpretable Deep Learning Method for Multiview Learning) to learn nonlinear relationships in data from multiple views while achieving feature selection. iDeepViewLearn combines deep learning flexibility with the statistical benefits of data and knowledge-driven feature selection, giving interpretable results. Deep neural networks are used to learn view-independent low-dimensional embedding through an optimization problem that minimizes the difference between observed and reconstructed data, while imposing a regularization penalty on the reconstructed data. The normalized Laplacian of a graph is used to model bilateral relationships between variables in each view, therefore, encouraging selection of related variables. iDeepViewLearn is tested on simulated and three real-world data for classification, clustering, and reconstruction tasks. For the classification tasks, iDeepViewLearn had competitive classification results with state-of-the-art methods in various settings. For the clustering task, we detected molecular clusters that differed in their 10-year survival rates for breast cancer. For the reconstruction task, we were able to reconstruct handwritten images using a few pixels while achieving competitive classification accuracy. The results of our real data application and simulations with small to moderate sample sizes suggest that iDeepViewLearn may be a useful method for small-sample-size problems compared to other deep learning methods for multiview learning.ConclusioniDeepViewLearn is an innovative deep learning model capable of capturing nonlinear relationships between data from multiple views while achieving feature selection. It is fully open source and is freely available at https://github.com/lasandrall/iDeepViewLearn .
| S-EPMC11265116 | biostudies-literature
Project description:Fluorescence-guided surgery is a state-of-the-art approach for intraoperative imaging during neurosurgical removal of tumor tissue. While the visualization of high-grade gliomas is reliable, lower grade glioma often lack visible fluorescence signals. Here, we present a hybrid prototype combining visible light optical coherence microscopy (OCM) and high-resolution fluorescence imaging for assessment of brain tumor samples acquired by 5-aminolevulinic acid (5-ALA) fluorescence-guided surgery. OCM provides high-resolution information of the inherent tissue scattering and absorption properties of tissue. We here explore quantitative attenuation coefficients derived from volumetric OCM intensity data and quantitative high-resolution 5-ALA fluorescence as potential biomarkers for tissue malignancy including otherwise difficult-to-assess low-grade glioma. We validate our findings against the gold standard histology and use attenuation and fluorescence intensity measures to differentiate between tumor core, infiltrative zone and adjacent brain tissue. Using large field-of-view scans acquired by a near-infrared swept-source optical coherence tomography setup, we provide initial assessments of tumor heterogeneity. Finally, we use cross-sectional OCM images to train a convolutional neural network that discriminates tumor from non-tumor tissue with an accuracy of 97%. Collectively, the present hybrid approach offers potential to translate into an in vivo imaging setup for substantially improved intraoperative guidance of brain tumor surgeries.
| S-EPMC7408054 | biostudies-literature
Project description:BackgroundSingle-cell multiplex imaging data have provided new insights into disease subtypes and prognoses recently. However, quantitative models that explicitly capture single-cell resolution cell-cell interaction features to predict patient survival at a population scale are currently missing.MethodsWe quantified hundreds of single-cell resolution cell-cell interaction features through neighborhood calculation, in addition to cellular phenotypes. We applied these features to a neural-network-based Cox-nnet survival model to identify survival-associated features. We used non-negative matrix factorization (NMF) to identify patient survival subtypes. We identified atypical subpopulations of triple-negative breast cancer (TNBC) patients with moderate prognosis and Luminal A patients with poor prognosis and validated these subpopulations by label transferring using the UNION-COM method.ResultsThe neural-network-based Cox-nnet survival model using all cellular phenotype and cell-cell interaction features is highly predictive of patient survival in the test data (Concordance Index > 0.8). We identify seven survival subtypes using the top survival features, presenting distinct profiles of epithelial, immune, and fibroblast cells and their interactions. We reveal atypical subpopulations of TNBC patients with moderate prognosis (marked by GATA3 over-expression) and Luminal A patients with poor prognosis (marked by KRT6 and ACTA2 over-expression and CDH1 under-expression). These atypical subpopulations are validated in TCGA-BRCA and METABRIC datasets.ConclusionsThis work provides an approach to bridge single-cell level information toward population-level survival prediction.
| S-EPMC10730890 | biostudies-literature
Project description:High-content cellular imaging, transcriptomics, and proteomics data provide rich and complementary views on the molecular layers of biology that influence cellular states and function. However, the biological determinants through which changes in multi-omics measurements influence cellular morphology have not yet been systematically explored, and the degree to which cell imaging could potentially enable the prediction of multi-omics directly from cell imaging data is therefore currently unclear. Here, we address the question of whether it is possible to predict bulk multi-omics measurements directly from cell images using Image2Omics -- a deep learning approach that predicts multi-omics in a cell population directly from high-content images stained with multiplexed fluorescent dyes. We perform an experimental evaluation in gene-edited macrophages derived from human induced pluripotent stem cell (hiPSC) under multiple stimulation conditions and demonstrate that Image2Omics achieves significantly better performance in predicting transcriptomics and proteomics measurements directly from cell images than predictors based on the mean observed training set abundance. We observed significant predictability of abundances for 5903 (22.43%; 95% CI: 8.77%, 38.88%) and 5819 (22.11%; 95% CI: 10.40%, 38.08%) transcripts out of 26137 in M1 and M2-stimulated macrophages respectively and for 1933 (38.77%; 95% CI: 36.94%, 39.85%) and 2055 (41.22%; 95% CI: 39.31%, 42.42%) proteins out of 4986 in M1 and M2-stimulated macrophages respectively. Our results show that some transcript and protein abundances are predictable from cell imaging and that cell imaging may potentially, in some settings and depending on the mechanisms of interest and desired performance threshold, even be a scalable and resource-efficient substitute for multi-omics measurements.
2025-07-15 | GSE237485 | GEO
Project description:BackgroundColorectal cancer (CRC) is a globally prevalent cancer. Emerging research implicates the gut microbiome in CRC pathogenesis. Bacteria such as Clostridium scindens can produce the carcinogenic bile acid deoxycholic acid (DCA). It is unknown whether imaging methods can differentiate DCA-producing and DCA-non-producing C. scindens cells.MethodsLight microscopy images of anaerobically cultured C. scindens in four conditions were acquired at 100× magnification using the Tissue FAX system: C. scindens in media alone (DCA-non-producing state), C. scindens in media with cholic acid (DCA-producing state), or C. scindens in co-culture with one of two Bacteroides species (intermediate DCA production states). We evaluated three approaches: whole-image classification, per-cell classification, and image segmentation-based classification. For whole-image classification, we used a custom Convolutional Neural Network (CNN), pre-trained DenseNet, pre-trained ResNet, and ResNet enhanced by integrating the Digital Images of Bacterial Species (DIBaS) dataset. For cell detection and classification, we applied thresholding (OTSU or adaptive thresholding) followed by a ResNet model. Finally, image segmentation-based classification was performed using nnU-Net.ResultsFor whole-image analysis, DIBaS-enhanced ResNet models achieved the best performance in distinguishing C. scindens states in monoculture (accuracy 0.89 ± 0.006) and in co-cultures (accuracy 0.86 ± 0.004). Per-cell analysis was optimal at a C constant value of 3, with the ResNet model achieving 62-74% accuracy for C. scindens states in monoculture. Segmentation-based analysis using nnU-Net resulted in Dice coefficients of 87% for C. scindens and 74-76% for the Bacteroides species.ConclusionsThis study demonstrates feasibility of image-based deep learning models in identifying health-relevant gut bacterial metabolic states.Supplementary informationThe online version contains supplementary material available at 10.1007/s44352-025-00006-1.
| S-EPMC11912549 | biostudies-literature
Project description:We propose a method for supervised learning with multiple sets of features ("views"). The multiview problem is especially important in biology and medicine, where "-omics" data, such as genomics, proteomics, and radiomics, are measured on a common set of samples. "Cooperative learning" combines the usual squared-error loss of predictions with an "agreement" penalty to encourage the predictions from different data views to agree. By varying the weight of the agreement penalty, we get a continuum of solutions that include the well-known early and late fusion approaches. Cooperative learning chooses the degree of agreement (or fusion) in an adaptive manner, using a validation set or cross-validation to estimate test set prediction error. One version of our fitting procedure is modular, where one can choose different fitting mechanisms (e.g., lasso, random forests, boosting, or neural networks) appropriate for different data views. In the setting of cooperative regularized linear regression, the method combines the lasso penalty with the agreement penalty, yielding feature sparsity. The method can be especially powerful when the different data views share some underlying relationship in their signals that can be exploited to boost the signals. We show that cooperative learning achieves higher predictive accuracy on simulated data and real multiomics examples of labor-onset prediction. By leveraging aligned signals and allowing flexible fitting mechanisms for different modalities, cooperative learning offers a powerful approach to multiomics data fusion.
| S-EPMC9499553 | biostudies-literature
Project description:Applying network science approaches to investigate the functions and anatomy of the human brain is prevalent in modern medical imaging analysis. Due to the complex network topology, for an individual brain, mining a discriminative network representation from the multimodal brain networks is non-trivial. The recent success of deep learning techniques on graph-structured data suggests a new way to model the non-linear cross-modality relationship. However, current deep brain network methods either ignore the intrinsic graph topology or require a network basis shared within a group. To address these challenges, we propose a novel end-to-end deep graph representation learning (Deep Multimodal Brain Networks - DMBN) to fuse multimodal brain networks. Specifically, we decipher the cross-modality relationship through a graph encoding and decoding process. The higher-order network mappings from brain structural networks to functional networks are learned in the node domain. The learned network representation is a set of node features that are informative to induce brain saliency maps in a supervised manner. We test our framework in both synthetic and real image data. The experimental results show the superiority of the proposed method over some other state-of-the-art deep brain network models.
| S-EPMC8293685 | biostudies-literature
Project description:With rapidly evolving high-throughput technologies and consistently decreasing costs, collecting multimodal omics data in large-scale studies has become feasible. Although studying multiomics provides a new comprehensive approach in understanding the complex biological mechanisms of human diseases, the high dimensionality of omics data and the complexity of the interactions among various omics levels in contributing to disease phenotypes present tremendous analytical challenges. There is a great need of novel analytical methods to address these challenges and to facilitate multiomics analyses. In this paper, we propose a multimodal functional deep learning (MFDL) method for the analysis of high-dimensional multiomics data. The MFDL method models the complex relationships between multiomics variants and disease phenotypes through the hierarchical structure of deep neural networks and handles high-dimensional omics data using the functional data analysis technique. Furthermore, MFDL leverages the structure of the multimodal model to capture interactions between different types of omics data. Through simulation studies and real-data applications, we demonstrate the advantages of MFDL in terms of prediction accuracy and its robustness to the high dimensionality and noise within the data.
| S-EPMC11405129 | biostudies-literature
Project description:PurposeTo examine deep learning (DL)-based methods for accurate segmentation of geographic atrophy (GA) lesions using fundus autofluorescence (FAF) and near-infrared (NIR) images.MethodsThis retrospective analysis utilized imaging data from study eyes of patients enrolled in Proxima A and B (NCT02479386; NCT02399072) natural history studies of GA. Two multimodal DL networks (UNet and YNet) were used to automatically segment GA lesions on FAF; segmentation accuracy was compared with annotations by experienced graders. The training data set comprised 940 image pairs (FAF and NIR) from 183 patients in Proxima B; the test data set comprised 497 image pairs from 154 patients in Proxima A. Dice coefficient scores, Bland-Altman plots, and Pearson correlation coefficient (r) were used to assess performance.ResultsOn the test set, Dice scores for the DL network to grader comparison ranged from 0.89 to 0.92 for screening visit; Dice score between graders was 0.94. GA lesion area correlations (r) for YNet versus grader, UNet versus grader, and between graders were 0.981, 0.959, and 0.995, respectively. Longitudinal GA lesion area enlargement correlations (r) for screening to 12 months (n = 53) were lower (0.741, 0.622, and 0.890, respectively) compared with the cross-sectional results at screening. Longitudinal correlations (r) from screening to 6 months (n = 77) were even lower (0.294, 0.248, and 0.686, respectively).ConclusionsMultimodal DL networks to segment GA lesions can produce accurate results comparable with expert graders.Translational relevanceDL-based tools may support efficient and individualized assessment of patients with GA in clinical research and practice.
| S-EPMC10341294 | biostudies-literature