MolE: a foundation model for molecular graphs using disentangled attention.

ABSTRACT: Models that accurately predict properties based on chemical structure are valuable tools in the chemical sciences. However, for many properties, public and private training sets are typically small, making it difficult for models to generalize well outside of the training data. Recently, this lack of generalization has been mitigated by using self-supervised pretraining on large unlabeled datasets, followed by finetuning on smaller, labeled datasets. Inspired by these advances, we report MolE, a Transformer architecture adapted for molecular graphs together with a two-step pretraining strategy. The first step of pretraining is a self-supervised approach focused on learning chemical structures trained on ~842 million molecular graphs, and the second step is a massive multi-task approach to

SUBMITTER: Mendez-Lucio O 

PROVIDER: S-EPMC11557931 | biostudies-literature | 2024 Nov

REPOSITORIES: biostudies-literature