Molecular causality in the advent of foundation models.
Ontology highlight
ABSTRACT:
SUBMITTER: Lobentanzer S
PROVIDER: S-EPMC11297329 | biostudies-literature | 2024 Jun
REPOSITORIES: biostudies-literature
Publications
Molecular causality in the advent of foundation models.
Lobentanzer Sebastian S Rodriguez-Mier Pablo P Bauer Stefan S Saez-Rodriguez Julio J
Molecular systems biology 20240618 8
Correlation is not causation: this simple and uncontroversial statement has far-reaching implications. Defining and applying causality in biomedical research has posed significant challenges to the scientific community. In this perspective, we attempt to connect the partly disparate fields of systems biology, causal reasoning, and machine learning to inform future approaches in the field of systems biology and molecular medicine. ...[more]
Similar Datasets
Project description:Pathology plays a crucial role in diagnosing and evaluating patient tissue samples obtained via surgeries and biopsies. The advent of whole slide scanners and the development of deep learning technologies have considerably advanced this field, promoting extensive research and development in pathology artificial intelligence (AI). These advancements have contributed to reduced workload of pathologists and supported decision-making in treatment plans. Large-scale AI models, known as foundation models (FMs), are more accurate and applicable to various tasks than traditional AI. Such models have recently emerged and expanded their application scope in healthcare. Numerous FMs have been developed in pathology, with reported applications in various tasks, such as disease and rare cancer diagnoses, patient survival prognosis prediction, biomarker expression prediction, and scoring of the immunohistochemical expression intensity. However, several challenges persist in the clinical application of FMs, which healthcare professionals, as users, must be aware of. Research to address these challenges is ongoing. In the future, the development of generalist medical AI, which integrates pathology FMs with FMs from other medical domains, is expected to progress, effectively utilizing AI in real clinical settings to promote precision and personalized medicine.
| S-EPMC11799676 | biostudies-literature
Project description:BackgroundThe 12-lead electrocardiogram (ECG) remains a cornerstone of cardiac diagnostics, yet existing artificial intelligence (AI) solutions for automated interpretation often lack generalizability, remain closed-source, and are primarily trained using supervised learning, limiting their adaptability across diverse clinical settings. To address these challenges, we developed and compared two open-source foundational ECG models: DeepECG-SSL, a self-supervised learning model, and DeepECG-SL, a supervised learning model.MethodsBoth models were trained on over 1 million ECGs using a standardized preprocessing pipeline and automated free-text extraction from ECG reports to predict 77 cardiac conditions. DeepECG-SSL was pretrained using self-supervised contrastive learning and masked lead modeling. The models were evaluated on six multilingual private healthcare systems and four public datasets for ECG interpretation across 77 diagnostic categories. Fairness analyses assessed disparities in performance across age and sex groups, while also investigating fairness and resource utilization.ResultsDeepECG-SSL achieved AUROCs of 0.990 (95%CI 0.990, 0.990) on internal dataset, 0.981 (95%CI 0.981, 0.981) on external public datasets, and 0.983 (95%CI 0.983, 0.983) on external private datasets, while DeepECG-SL demonstrated AUROCs of 0.992 (95%CI 0.992, 0.992), 0.980 (95%CI 0.980, 0.980) and 0.983 (95%CI 0.983, 0.983) respectively. Fairness analyses revealed minimal disparities (true positive rate & false positive rate difference<0.010) across age and sex groups. Digital biomarker prediction (Long QT syndrome (LQTS) classification, 5-year atrial fibrillation prediction and left ventricular ejection fraction (LVEF) classification) with limited labeled data, DeepECG-SSL outperformed DeepECG-SL in predicting 5-year atrial fibrillation risk (N=132,050; AUROC 0.742 vs. 0.720; Δ=0.022; P<0.001), identifying reduced LVEF ≤40% (N=25,252; 0.928 vs. 0.900; Δ=0.028; P<0.001), and classifying LQTS syndrome subtypes (N=127; 0.931 vs. 0.853; Δ=0.078; P=0.026).ConclusionBy releasing model weights, preprocessing tools, and validation code, we aim to support robust, data-efficient AI diagnostics across diverse clinical environments. This study establishes self-supervised learning as a promising paradigm for ECG analysis, particularly in settings with limited annotated data, enhancing accessibility, generalizability, and fairness in AI-driven cardiac diagnostics.
| S-EPMC11908279 | biostudies-literature
Project description:In the last decades, Systems Biology (including cancer research) has been driven by technology, statistical modelling and bioinformatics. In this paper we try to bring biological and philosophical thinking back. We thus aim at making different traditions of thought compatible: (a) causality in epidemiology and in philosophical theorizing-notably, the "sufficient-component-cause framework" and the "mark transmission" approach; (b) new acquisitions about disease pathogenesis, e.g. the "branched model" in cancer, and the role of biomarkers in this process; (c) the burgeoning of omics research, with a large number of "signals" and of associations that need to be interpreted. In the paper we summarize first the current views on carcinogenesis, and then explore the relevance of current philosophical interpretations of "cancer causes". We try to offer a unifying framework to incorporate biomarkers and omic data into causal models, referring to a position called "evidential pluralism". According to this view, causal reasoning is based on both "evidence of difference-making" (e.g. associations) and on "evidence of underlying biological mechanisms". We conceptualize the way scientists detect and trace signals in terms of information transmission, which is a generalization of the mark transmission theory developed by philosopher Wesley Salmon. Our approach is capable of helping us conceptualize how heterogeneous factors such as micro and macro-biological and psycho-social-are causally linked. This is important not only to understand cancer etiology, but also to design public health policies that target the right causal factors at the macro-level.
| S-EPMC5463386 | biostudies-literature
Project description:The rapid advancement of DNA foundation language models has revolutionized the field of genomics, enabling the decoding of complex patterns and regulatory mechanisms within DNA sequences. However, the current evaluation of these models often relies on fine-tuning and limited datasets, which introduces biases and limits the assessment of their true potential. Here, we present a benchmarking study of three recent DNA foundation language models, including DNABERT-2, Nucleotide Transformer version-2 (NT-v2), and HyenaDNA, focusing on the quality of their zero-shot embeddings across a diverse range of genomic tasks and species through analyses of 57 real datasets. We found that DNABERT-2 exhibits the most consistent performance across human genome-related tasks, while NT-v2 excels in epigenetic modification detection. HyenaDNA stands out for its exceptional runtime scalability and ability to handle long input sequences. Importantly, we demonstrate that using mean token embedding consistently improves the performance of all three models compared to the default setting of sentence-level summary token embedding, with average AUC improvements ranging from 4.3% to 9.7% for different DNA foundation models. Furthermore, the performance differences between these models are significantly reduced when using mean token embedding. Our findings provide a framework for selecting and optimizing DNA language models, guiding researchers in applying these tools effectively in genomic studies.
| S-EPMC11343214 | biostudies-literature
Project description:Vision impairment affects nearly 2.2 billion people globally, and nearly half of these cases could be prevented with early diagnosis and intervention-underscoring the urgent need for reliable and scalable detection methods for conditions like diabetic retinopathy and age-related macular degeneration. Here we propose a distributed deep learning framework that integrates self-supervised and domain-adaptive federated learning to enhance the detection of eye diseases from optical coherence tomography images. We employed a self-supervised, mask-based pre-training strategy to develop a robust foundation encoder. This encoder was trained on seven optical coherence tomography datasets, and we compared its performance under local, centralized, and federated learning settings. Our results show that self-supervised methods-both centralized and federated-improved the area under the curve by at least 10% compared to local models. Additionally, incorporating domain adaptation into the federated learning framework further boosted performance and generalization across different populations and imaging conditions. This approach supports collaborative model development without data sharing, providing a scalable, privacy-preserving solution for effective retinal disease screening and diagnosis in diverse clinical settings.
| S-EPMC11754456 | biostudies-literature
Project description:MotivationHydrophobic patches on protein surfaces play important functional roles in protein-protein and protein-ligand interactions. Large hydrophobic surfaces are also involved in the progression of aggregation diseases. Predicting exposed hydrophobic patches from a protein sequence has shown to be a difficult task. Fine-tuning foundation models allows for adapting a model to the specific nuances of a new task using a much smaller dataset. Additionally, multitask deep learning offers a promising solution for addressing data gaps, simultaneously outperforming single-task methods.ResultsIn this study, we harnessed a recently released leading large language model Evolutionary Scale Models (ESM-2). Efficient fine-tuning of ESM-2 was achieved by leveraging a recently developed parameter-efficient fine-tuning method. This approach enabled comprehensive training of model layers without excessive parameters and without the need to include a computationally expensive multiple sequence analysis. We explored several related tasks, at local (residue) and global (protein) levels, to improve the representation of the model. As a result, our model, PatchProt, cannot only predict hydrophobic patch areas but also outperforms existing methods at predicting primary tasks, including secondary structure and surface accessibility predictions. Importantly, our analysis shows that including related local tasks can improve predictions on more difficult global tasks. This research sets a new standard for sequence-based protein property prediction and highlights the remarkable potential of fine-tuning foundation models enriching the model representation by training over related tasks.Availability and implementationhttps://github.com/Deagogishvili/chapter-multi-task.
| S-EPMC11525051 | biostudies-literature
Project description:We evaluated several foundation models in histopathology for image retrieval using a zero-shot approach. These models generated embeddings that were directly employed for retrieval without additional fine-tuning. Our experiments were conducted on diagnostic slides from The Cancer Genome Atlas (TCGA), which covers 23 organs and 117 cancer subtypes. We used Yottixel as the framework for whole-slide image (WSI) retrieval via patch-based embeddings. Retrieval performance was evaluated using macro-averaged F1 scores for top-1, top-3, and top-5 retrievals. The top-5 retrieval F1 scores indicated varying levels of performance: Yottixel-DenseNet (27% ± 13%), Yottixel-UNI (42% ± 14%), Yottixel-Virchow (40% ± 13%), Yottixel-GigaPath (41% ± 13%), and GigaPath WSI (40% ± 14%). These results demonstrate the potential and limitations of foundation models for histopathology image retrieval, underscoring the need for further advancements in embedding and retrieval techniques.
| S-EPMC11787325 | biostudies-literature
Project description:Generative adversarial networks (GANs), first published in 2014, are among the most important concepts in modern artificial intelligence (AI). Bridging deep learning and game theory, GANs are used to generate or "imagine" new objects with desired properties. Since 2016, multiple GANs with reinforcement learning (RL) have been successfully applied in pharmacology for de novo molecular design. Those techniques aim at a more efficient use of the data and a better exploration of the chemical space. We review recent advances for the generation of novel molecules with desired properties with a focus on the applications of GANs, RL, and related techniques. We also discuss the current limitations and challenges in the new growing field of generative chemistry.
| S-EPMC7429972 | biostudies-literature
Project description:In the canonical genetic code, many amino acids are assigned more than one codon. Work by us and others has shown that the choice of these synonymous codon is not random, and carries regulatory and functional consequences. Existing protein foundation models ignore this context-dependent role of coding sequence in shaping the protein landscape of the cell. To address this gap, we introduce cdsFM, a suite of codon-resolution large language models, including both EnCodon and DeCodon models, with up to 1B parameters. Pre-trained on 60 million protein-coding sequences from more than 5,000 species, our models effectively learn the relationship between codons and amino acids, recapitualing the overall structure of the genetic code. In addition to outperforming state-of-the-art genomic foundation models in a variety of zero-shot and few-shot learning tasks, the larger pre-trained models were superior in predicting the choice of synonymous codons. To systematically assess the impact of synonymous codon choices on protein expression and our models' ability to capture these effects, we generated a large dataset measuring overall and surface expression levels of three proteins as a function of changes in their synonymous codons. We showed that our EnCodon models could be readily fine-tuned to predict the contextual consequences of synonymous codon choices. Armed with this knowledge, we applied EnCodon to existing clinical datasets of synonymous variants, and we identified a large number of synonymous codons that are likely pathogenic, several of which we experimentally confirmed in a cell-based model. Together, our findings establish the cdsFM suite as a powerful tool for decoding the complex functional grammar underlying the choice of synonymous codons.
| S-EPMC11482952 | biostudies-literature
Project description:Recent advancements in large language models (LLMs) show significant potential in medical applications but are hindered by limited specialized medical knowledge. We present Me-LLaMA, a family of open-source medical LLMs integrating extensive domain-specific knowledge with robust instruction-following capabilities. Me-LLaMA is developed through continual pretraining and instruction tuning of LLaMA2 models using diverse biomedical and clinical data sources (e.g., biomedical literature and clinical notes). We evaluated Me-LLaMA on six text analysis tasks using 12 benchmarks (e.g., PubMedQA and MIMIC-CXR) and assessed its clinical utility in complex case diagnosis through automatic and human evaluations. Me-LLaMA outperforms existing open medical LLMs in zero-shot and supervised settings and surpasses ChatGPT and GPT-4 after task-specific instruction tuning for most text analysis tasks. Its performance is also comparable to ChatGPT and GPT-4 for diagnosing complex clinical cases. Our findings highlight the importance of combining domain-specific continual pretraining with instruction tuning to enhance performance in medical LLMs.
| S-EPMC11882967 | biostudies-literature