Generative language models on nucleotide sequences of human genes.

ABSTRACT: Language models, especially transformer-based ones, have achieved colossal success in natural language processing. To be precise, studies like BERT for natural language understanding and works like GPT-3 for natural language generation are very important. If we consider DNA sequences as a text written with an alphabet of four letters representing the nucleotides, they are similar in structure to natural languages. This similarity has led to the development of discriminative language models such as DNABERT in the field of DNA-related bioinformatics. To our knowledge, however, the generative side of the coin is still largely unexplored. Therefore, we have focused on the development of an autoregressive generative language model such as GPT-3 for DNA sequences. Since working with whole DNA se

SUBMITTER: Ihtiyar MN 

PROVIDER: S-EPMC11437190 | biostudies-literature | 2024 Sep

REPOSITORIES: biostudies-literature