The Data Efficiency of Deep Learning Is Degraded by Unnecessary Input Dimensions.

ABSTRACT: Biological learning systems are outstanding in their ability to learn from limited training data compared to the most successful learning machines, i.e., Deep Neural Networks (DNNs). What are the key aspects that underlie this data efficiency gap is an unresolved question at the core of biological and artificial intelligence. We hypothesize that one important aspect is that biological systems rely on mechanisms such as foveations in order to reduce unnecessary input dimensions for the task at hand, e.g., background in object recognition, while state-of-the-art DNNs do not. Datasets to train DNNs often contain such unnecessary input dimensions, and these lead to more trainable parameters. Yet, it is not clear whether this affects the DNNs' data efficiency because DNNs are robu

SUBMITTER: D'Amario V 

PROVIDER: S-EPMC8842477 | biostudies-literature | 2022

REPOSITORIES: biostudies-literature