Isometric representations in neural networks improve robustness.

ABSTRACT: Artificial and biological agents are unable to learn given completely random and unstructured data. The structure of data is encoded in the distance or similarity relationships between data points. In the context of neural networks, the neuronal activity within a layer forms a representation reflecting the transformation that the layer implements on its inputs. In order to utilize the structure in the data in a truthful manner, such representations should reflect the input distances and thus be continuous and isometric. Supporting this statement, findings in neuroscience propose that generalization and robustness are tied to neural representations being continuously differentiable. Furthermore, representations of objects have the capacity of being hierarchical. Combined together, these two

SUBMITTER: Beshkov K 

PROVIDER: S-EPMC12541071 | biostudies-literature | 2025 Oct

REPOSITORIES: biostudies-literature