An Information Theoretic Approach to Reveal the Formation of Shared Representations.

ABSTRACT: Modality-invariant categorical representations, i.e., shared representation, is thought to play a key role in learning to categorize multi-modal information. We have investigated how a bimodal autoencoder can form a shared representation in an unsupervised manner with multi-modal data. We explored whether altering the depth of the network and mixing the multi-modal inputs at the input layer affect the development of the shared representations. Based on the activation of units in the hidden layers, we classified them into four different types: visual cells, auditory cells, inconsistent visual and auditory cells, and consistent visual and auditory cells. Our results show that the number and quality of the last type (i.e., shared representation) significantly differ depending on the depth of

SUBMITTER: Eguchi A 

PROVIDER: S-EPMC7001587 | biostudies-literature | 2020

REPOSITORIES: biostudies-literature