Locality-aware pooling enhances protein language model performance across varied applications.

ABSTRACT:

Motivation

Protein language models (PLMs) are amongst the most exciting recent advances for characterizing protein sequences, and have enabled a diverse set of applications, including structure determination, functional property prediction, and mutation impact assessment, all from single protein sequences alone. State-of-the-art PLMs leverage transformer architectures originally developed for natural language processing, and are pre-trained on large protein databases to generate contextualized representations of individual amino acids. To harness the power of these PLMs to predict protein-level properties, these per-residue embeddings are typically "pooled" to fixed-size vectors that are further utilized in downstream prediction networks. Common pooling strategies include Cls-Pooli

SUBMITTER: Hoang M 

PROVIDER: S-EPMC12261463 | biostudies-literature | 2025 Jul

REPOSITORIES: biostudies-literature