BERT2DAb: a pre-trained model for antibody representation based on amino acid sequences and 2D-structure.

ABSTRACT: Prior research has generated a vast amount of antibody sequences, which has allowed the pre-training of language models on amino acid sequences to improve the efficiency of antibody screening and optimization. However, compared to those for proteins, there are fewer pre-trained language models available for antibody sequences. Additionally, existing pre-trained models solely rely on embedding representations using amino acids or k-mers, which do not explicitly take into account the role of secondary structure features. Here, we present a new pre-trained model called BERT2DAb. This model incorporates secondary structure information based on self-attention to learn representations of antibody sequences. Our model achieves state-of-the-art performance on three downstream tasks, including two

SUBMITTER: Luo X 

PROVIDER: S-EPMC10793684 | biostudies-literature | 2023 Jan-Dec

REPOSITORIES: biostudies-literature