Discriminative, Restorative, and Adversarial Learning: Stepwise Incremental Pretraining

ABSTRACT: Uniting three self-supervised learning (SSL) ingredients (discriminative, restorative, and adversarial learning) enables collaborative representation learning and yields three transferable components: a discriminative encoder, a restorative decoder, and an adversary encoder. To leverage this advantage, we have redesigned five prominent SSL methods, including Rotation, Jigsaw, Rubik’s Cube, Deep Clustering, and TransVW, and formulated each in a United framework for 3D medical imaging. However, such a United framework increases model complexity and pretraining difficulty. To overcome this difficulty, we develop a stepwise incremental pretraining strategy, in which a discriminative encoder is first trained via discriminative learning, the pretrained discriminative encoder is then attached to

SUBMITTER: Guo Z 

PROVIDER: S-EPMC9728134 | biostudies-literature | 2022 Sep

REPOSITORIES: biostudies-literature