Medium-sized protein language models perform well at transfer learning on realistic datasets.

ABSTRACT: Protein language models (pLMs) can offer deep insights into evolutionary and structural properties of proteins. While larger models, such as the 15 billion parameter model ESM-2, promise to capture more complex patterns in sequence space, they also present practical challenges due to their high dimensionality and high computational cost. We systematically evaluated the performance of various ESM-style models across multiple biological datasets to assess the impact of model size on transfer learning via feature extraction. Surprisingly, we found that larger models do not necessarily outperform smaller ones, in particular when data is limited. Medium-sized models, such as ESM-2 650M and ESM C 600M, demonstrated consistently good performance, falling only slightly behind their larger counterp

SUBMITTER: Vieira LC 

PROVIDER: S-EPMC12217344 | biostudies-literature | 2025 Jul

REPOSITORIES: biostudies-literature