A Manifold Learning Perspective on Representation Learning: Learning Decoder and Representations without an Encoder.

ABSTRACT: Autoencoders are commonly used in representation learning. They consist of an encoder and a decoder, which provide a straightforward method to map n-dimensional data in input space to a lower m-dimensional representation space and back. The decoder itself defines an m-dimensional manifold in input space. Inspired by manifold learning, we showed that the decoder can be trained on its own by learning the representations of the training samples along with the decoder weights using gradient descent. A sum-of-squares loss then corresponds to optimizing the manifold to have the smallest Euclidean distance to the training samples, and similarly for other loss functions. We derived expressions for the number of samples needed to specify the encoder and decoder and showed that the decoder generally

SUBMITTER: Schuster V 

PROVIDER: S-EPMC8625121 | biostudies-literature | 2021 Oct

REPOSITORIES: biostudies-literature