Biologically Plausible Training Mechanisms for Self-Supervised Learning in Deep Networks.

ABSTRACT: We develop biologically plausible training mechanisms for self-supervised learning (SSL) in deep networks. Specifically, by biologically plausible training we mean (i) all updates of weights are based on current activities of pre-synaptic units and current, or activity retrieved from short term memory of post synaptic units, including at the top-most error computing layer, (ii) complex computations such as normalization, inner products and division are avoided, (iii) asymmetric connections between units, and (iv) most learning is carried out in an unsupervised manner. SSL with a contrastive loss satisfies the third condition as it does not require labeled data and it introduces robustness to observed perturbations of objects, which occur naturally as objects or observers move in 3D

SUBMITTER: Tang M 

PROVIDER: S-EPMC8977509 | biostudies-literature | 2022

REPOSITORIES: biostudies-literature