{"database":"biostudies-literature","file_versions":[],"scores":null,"additional":{"submitter":["An J"],"funding":["Innovative Research Team of Northeast Agricultural University"],"pagination":["467"],"full_dataset_link":["https://www.ebi.ac.uk/biostudies/studies/S-EPMC9641823"],"repository":["biostudies-literature"],"omics_type":["Unknown"],"volume":["23(1)"],"pubmed_abstract":["Pre-trained natural language processing models on a large natural language corpus can naturally transfer learned knowledge to protein domains by fine-tuning specific in-domain tasks. However, few studies focused on enriching such protein language models by jointly learning protein properties from strongly-correlated protein tasks. Here we elaborately designed a multi-task learning (MTL) architecture, aiming to decipher implicit structural and evolutionary information from three sequence-level classification tasks for protein family, superfamily and fold. Considering the co-existing contextual relevance between human words and protein language, we employed BERT, pre-trained on a large natural language corpus, as our backbone to handle protein sequences. More importantly, the encoded knowled"],"journal":["BMC bioinformatics"],"pubmed_title":["Collectively encoding protein properties enriches protein language models."],"pmcid":["PMC9641823"],"funding_grant_id":["No. 54941012"],"pubmed_authors":["Weng X","An J"],"additional_accession":[]},"is_claimable":false,"name":"Collectively encoding protein properties enriches protein language models.","description":"Pre-trained natural language processing models on a large natural language corpus can naturally transfer learned knowledge to protein domains by fine-tuning specific in-domain tasks. However, few studies focused on enriching such protein language models by jointly learning protein properties from strongly-correlated protein tasks. Here we elaborately designed a multi-task learning (MTL) architecture, aiming to decipher implicit structural and evolutionary information from three sequence-level classification tasks for protein family, superfamily and fold. Considering the co-existing contextual relevance between human words and protein language, we employed BERT, pre-trained on a large natural language corpus, as our backbone to handle protein sequences. More importantly, the encoded knowled","dates":{"release":"2022-01-01T00:00:00Z","publication":"2022 Nov","modification":"2026-04-08T11:37:05.731Z","creation":"2025-04-19T22:46:22.369Z"},"accession":"S-EPMC9641823","cross_references":{"pubmed":["36348281"],"doi":["10.1186/s12859-022-05031-z"]}}