Paying attention to attention: High attention sites as indicators of protein family and function in language models.

ABSTRACT: Protein Language Models (PLMs) use transformer architectures to capture patterns within protein primary sequences, providing a powerful computational representation of the amino acid sequence. Through large-scale training on protein primary sequences, PLMs generate vector representations that encapsulate the biochemical and structural properties of proteins. At the core of PLMs is the attention mechanism, which facilitates the capture of long-range dependencies by computing pairwise importance scores across residues, thereby highlighting regions of biological interaction within the sequence. The attention matrices offer an untapped opportunity to uncover specific biological properties of proteins, particularly their functions. In this work, we introduce a novel approach, using the Evolutio

SUBMITTER: Nayar G 

PROVIDER: S-EPMC12448987 | biostudies-literature | 2025 Sep

REPOSITORIES: biostudies-literature