GenSLMs: Genome-scale language models reveal SARS-CoV-2 evolutionary dynamics.
Ontology highlight
ABSTRACT:
SUBMITTER: Zvyagin M
PROVIDER: S-EPMC9709791 | biostudies-literature | 2022 Nov
REPOSITORIES: biostudies-literature
Publications
GenSLMs: Genome-scale language models reveal SARS-CoV-2 evolutionary dynamics.
Zvyagin Maxim M Brace Alexander A Hippe Kyle K Deng Yuntian Y Zhang Bin B Bohorquez Cindy Orozco CO Clyde Austin A Kale Bharat B Perez-Rivera Danilo D Ma Heng H Mann Carla M CM Irvin Michael M Pauloski J Gregory JG Ward Logan L Hayot-Sasson Valerie V Emani Murali M Foreman Sam S Xie Zhen Z Lin Diangen D Shukla Maulik M Nie Weili W Romero Josh J Dallago Christian C Vahdat Arash A Xiao Chaowei C Gibbs Thomas T Foster Ian I Davis James J JJ Papka Michael E ME Brettin Thomas T Stevens Rick R Anandkumar Anima A Vishwanath Venkatram V Ramanathan Arvind A
bioRxiv : the preprint server for biology 20221123
We seek to transform how new and emergent variants of pandemic-causing viruses, specifically SARS-CoV-2, are identified and classified. By adapting large language models (LLMs) for genomic data, we build genome-scale language models (GenSLMs) which can learn the evolutionary landscape of SARS-CoV-2 genomes. By pre-training on over 110 million prokaryotic gene sequences and fine-tuning a SARS-CoV-2-specific model on 1.5 million genomes, we show that GenSLMs can accurately and rapidly identify var ...[more]
Similar Datasets
Project description:Models of evolution of simple languages have typically assumed full alignment of the speaker and listeners interests, with perfect understanding representing the optimal outcome for both parties. In more realistic settings, communicating individuals will often desire different outcomes from one another. Previous work has shown that misalignment of speaker-listener interests reduces the maximum informativeness among Nash-equilibrium languages, and that multiple equilibrium languages (with different degrees of informativeness) are supported. We study the stochastic evolutionary dynamics of signaling games in which the alignment of speaker-listener interests can vary. We find that increased misalignment of speaker-listener interests is associated with a decrease in information transmission. Moreover, the most common languages to evolve are typically the most informative languages supportable as static Nash equilibria, suggesting a solution to the ‘equilibrium selection problem’. In addition, our dynamics reveal the mechanism by which less informative languages evolve: words that previously signaled intense states come to be used hyperbolically for less intense states, with listeners’ interpretation of these newly-ambiguous words evolving downward in response. We ground our results in linguistic data on intensifiers such as so and very, words which have unique dynamics—with constant recycling and innovation that match our theoretical results well. Author summary Analyses of signaling and language have typically assumed that both speakers and listeners have a common interest, such as to communicate the location of predators or resources or to convey precise instructions on how to perform a particular task. However, Nature is also rife with situations where mutual interest cannot be assumed. Many such situations have gradated states, signals, and actions, where speaker and listener desire different actions in conjunction with different true states, which are only known to the speaker. For instance, baby birds regularly overstate their true hunger level to their mother, due to a drive to obtain more than the base requirements in attention and nutrients; there are countless parallels in the context of human communication. Previous works in static game theory have established interesting results in modeling misaligned talk, such as the presence of multiple equilibria and the rapid shrinkage of lexica as misalignment of interest increases. Here we expand both the static theory, but also show using evolutionary game theory which equilibria are selected and why. We make a strong connection to linguistics via intensifiers—words such as “so”, “really”, and “very”—which suggests this literature may be immediately salient to sociolinguistics and other disciplines.
| S-EPMC9949666 | biostudies-literature
Project description:Understanding how and why language subsystems differ in their evolutionary dynamics is a fundamental question for historical and comparative linguistics. One key dynamic is the rate of language change. While it is commonly thought that the rapid rate of change hampers the reconstruction of deep language relationships beyond 6,000-10,000 y, there are suggestions that grammatical structures might retain more signal over time than other subsystems, such as basic vocabulary. In this study, we use a Dirichlet process mixture model to infer the rates of change in lexical and grammatical data from 81 Austronesian languages. We show that, on average, most grammatical features actually change faster than items of basic vocabulary. The grammatical data show less schismogenesis, higher rates of homoplasy, and more bursts of contact-induced change than the basic vocabulary data. However, there is a core of grammatical and lexical features that are highly stable. These findings suggest that different subsystems of language have differing dynamics and that careful, nuanced models of language change will be needed to extract deeper signal from the noise of parallel evolution, areal readaptation, and contact.
| S-EPMC5651730 | biostudies-literature
Project description:The coronavirus disease 2019 (COVID-19) pandemic, caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), is a major public health concern. A handful of static structures now provide molecular insights into how SARS-CoV-2 and SARS-CoV interact with its host target, which is the angiotensin converting enzyme 2 (ACE2). Molecular recognition, binding and function are dynamic processes. To evaluate this, multiple 500 ns or 1 μs all-atom molecular dynamics simulations were performed to better understand the structural stability and interfacial interactions between the receptor binding domain of the spike (S) protein of SARS-CoV-2 and SARS-CoV bound to ACE2. Several contacts were observed to form, break and reform in the interface during the simulations. Our results indicate that SARS-CoV-2 and SARS-CoV utilizes unique strategies to achieve stable binding to ACE2. Several differences were observed between the residues of SARS-CoV-2 and SARS-CoV that consistently interacted with ACE2. Notably, a stable salt bridge between Lys417 of SARS-CoV-2 S protein and Asp30 of ACE2 as well as three stable hydrogen bonds between Tyr449, Gln493 and Gln498 of SARS-CoV-2 and Asp38, Glu35 and Lys353 of ACE2 were observed, which were absent in the ACE2-SARS-CoV interface. Some previously reported residues, which were suggested to enhance the binding affinity of SARS-CoV-2, were not observed to form stable interactions in these simulations. Molecular mechanics-generalized Born surface area based free energy of binding was observed to be higher for SARS-CoV-2 in all simulations. Stable binding to the host receptor is crucial for virus entry. Therefore, special consideration should be given to these stable interactions while designing potential drugs and treatment modalities to target or disrupt this interface.
| S-EPMC7449962 | biostudies-literature
Project description:The new SARS-CoV-2 poses a significant threat to human health but many aspects of its basic biology remain unknown. Its genome encodes accessory genes that differ significantly within coronaviruses and contribute to the virus pathogenicity. Among accessory genes, open reading frame 8 (ORF8) stands out by being highly variable and showing structural changes suspected to be related with the virus ability to spread. However, the function of ORF8 remains to be elucidated, making it less studied than other SARS-CoV-2 genes. Here I show that ORF8 is poorly conserved among related coronaviruses. The ORF8 phylogeny built using 11,113 SARS-CoV-2 sequences revealed traces of a typical expanding population with a small number of highly frequent lineages. Interestingly, I detected several nonsense mutations and three main deletions in the ORF8 gene that either remove or significantly change the ORF8 protein. These findings suggest that SARS-CoV-2 can persist without a functional ORF8 protein. Deletion breakpoints were found located in predicted hairpins suggesting a possible involvement of these elements in the rearrangement process. Although the function of ORF8 remains to be elucidated, its structural plasticity and high diversity suggest an important role in SARS-CoV-2 pathogenicity.
| S-EPMC7467077 | biostudies-literature
Project description:SARS-CoV-2, responsible for the current COVID-19 pandemic that claimed over 5.0 million lives, belongs to a class of enveloped viruses that undergo quick evolutionary adjustments under selection pressure. Numerous variants have emerged in SARS-CoV-2, posing a serious challenge to the global vaccination effort and COVID-19 management. The evolutionary dynamics of this virus are only beginning to be explored. In this work, we have analysed 1.79 million spike glycoprotein sequences of SARS-CoV-2 and found that the virus is fine-tuning the spike with numerous amino acid insertions and deletions (indels). Indels seem to have a selective advantage as the proportions of sequences with indels steadily increased over time, currently at over 89%, with similar trends across countries/variants. There were as many as 420 unique indel positions and 447 unique combinations of indels. Despite their high frequency, indels resulted in only minimal alteration of N-glycosylation sites, including both gain and loss. As indels and point mutations are positively correlated and sequences with indels have significantly more point mutations, they have implications in the evolutionary dynamics of the SARS-CoV-2 spike glycoprotein.
| S-EPMC8655444 | biostudies-literature
Project description:Heritable trait variation is a central and necessary ingredient of evolution. Trait variation also directly affects ecological processes, generating a clear link between evolutionary and ecological dynamics. Despite the changes in variation that occur through selection, drift, mutation, and recombination, current eco-evolutionary models usually fail to track how variation changes through time. Moreover, eco-evolutionary models assume fitness functions for each trait and each ecological context, which often do not have empirical validation. We introduce a new type of model, Gillespie eco-evolutionary models (GEMs), that resolves these concerns by tracking distributions of traits through time as eco-evolutionary dynamics progress. This is done by allowing change to be driven by the direct fitness consequences of model parameters within the context of the underlying ecological model, without having to assume a particular fitness function. GEMs work by adding a trait distribution component to the standard Gillespie algorithm - an approach that models stochastic systems in nature that are typically approximated through ordinary differential equations. We illustrate GEMs with the Rosenzweig-MacArthur consumer-resource model. We show not only how heritable trait variation fuels trait evolution and influences eco-evolutionary dynamics, but also how the erosion of variation through time may hinder eco-evolutionary dynamics in the long run. GEMs can be developed for any parameter in any ordinary differential equation model and, furthermore, can enable modeling of multiple interacting traits at the same time. We expect GEMs will open the door to a new direction in eco-evolutionary and evolutionary modeling by removing long-standing modeling barriers, simplifying the link between traits, fitness, and dynamics, and expanding eco-evolutionary treatment of a greater diversity of ecological interactions. These factors make GEMs much more than a modeling advance, but an important conceptual advance that bridges ecology and evolution through the central concept of heritable trait variation.
| S-EPMC4761774 | biostudies-literature
Project description:Human language is based on grammatical rules. Cultural evolution allows these rules to change over time. Rules compete with each other: as new rules rise to prominence, old ones die away. To quantify the dynamics of language evolution, we studied the regularization of English verbs over the past 1,200 years. Although an elaborate system of productive conjugations existed in English's proto-Germanic ancestor, Modern English uses the dental suffix, '-ed', to signify past tense. Here we describe the emergence of this linguistic rule amidst the evolutionary decay of its exceptions, known to us as irregular verbs. We have generated a data set of verbs whose conjugations have been evolving for more than a millennium, tracking inflectional changes to 177 Old-English irregular verbs. Of these irregular verbs, 145 remained irregular in Middle English and 98 are still irregular today. We study how the rate of regularization depends on the frequency of word usage. The half-life of an irregular verb scales as the square root of its usage frequency: a verb that is 100 times less frequent regularizes 10 times as fast. Our study provides a quantitative analysis of the regularization process by which ancestral forms gradually yield to an emerging linguistic rule.
| S-EPMC2460562 | biostudies-literature
Project description:While the genome is composed of individual nucleotides, functional elements such as cis-regulatory elements and structural interactions are formed from sets of interdependent nucleotides. In principle, these dependencies are reflected in coevolutionary relationships. However, classical comparative genomics approaches struggle to detect these dependencies beyond alignable highly conserved sequences such as within coding regions. DNA language models (LMs), which are trained by predicting nucleotides given their sequence context, have recently been proposed as foundational models for sequence-based prediction problems. DNA LMs implicitly capture functional elements from genomic sequences alone. However, which dependencies DNA LMs learn and whether they reflect known or even novel biology remains an open question. Here we introduce nucleotide dependency maps to systematically study nucleotide dependencies captured by DNA LMs in a purely unsupervised setup. We compute these maps genome-wide and show that they reveal and clearly delineate known functional genomic features such as transcription factor binding motifs, functional interactions between splice sites, RNA tertiary structures, and coding sequences. This allowed to uncover novel experimentally validated RNA structures. We furthermore investigate dependency maps from in silico manipulated sequences, revealing the ability of DNA LMs to capture operations such as copying and reverse complementarity without memorization. Lastly, we compare dependency maps from openly available DNA LMs, showcasing the drawbacks and advantages of different models. We find stark differences in the ability of models to accurately learn conserved but infrequent features. Altogether, by leveraging the flexibility of DNA language models, nucleotide dependency mapping emerges as a general methodology to discover and study functional interactions in genomes.
2025-07-19 | GSE271937 | GEO
Project description:Antibodies are proteins produced by the immune system that can identify and neutralise a wide variety of antigens with high specificity and affinity, and constitute the most successful class of biotherapeutics. With the advent of next-generation sequencing, billions of antibody sequences have been collected in recent years, though their application in the design of better therapeutics has been constrained by the sheer volume and complexity of the data. To address this challenge, we present IgBert and IgT5, the best performing antibody-specific language models developed to date which can consistently handle both paired and unpaired variable region sequences as input. These models are trained comprehensively using the more than two billion unpaired sequences and two million paired sequences of light and heavy chains present in the Observed Antibody Space dataset. We show that our models outperform existing antibody and protein language models on a diverse range of design and regression tasks relevant to antibody engineering. This advancement marks a significant leap forward in leveraging machine learning, large scale data sets and high-performance computing for enhancing antibody design for therapeutic development.
| S-EPMC11654935 | biostudies-literature
Project description:To assess dynamics of SARS-CoV-2 in Greater Accra Region, Ghana, we analyzed SARS-CoV-2 genomic sequences from persons in the community and returning from international travel. The Accra Metropolitan District was a major origin of virus spread to other districts and should be a primary focus for interventions against future infectious disease outbreaks.
| S-EPMC10045712 | biostudies-literature