Universal Hopfield Networks: A General Framework for Single-Shot Associative Memory Models.

ABSTRACT: A large number of neural network models of associative memory have been proposed in the literature. These include the classical Hopfield networks (HNs), sparse distributed memories (SDMs), and more recently the modern continuous Hopfield networks (MCHNs), which possess close links with self-attention in machine learning. In this paper, we propose a general framework for understanding the operation of such memory networks as a sequence of three operations: similarity, separation, and projection. We derive all these memory models as instances of our general framework with differing similarity and separation functions. We extend the mathematical framework of Krotov & Hopfield (2020) to express general associative memory models using neural network dynamics with local comp

SUBMITTER: Millidge B 

PROVIDER: S-EPMC7614148 | biostudies-literature | 2022 Jul

REPOSITORIES: biostudies-literature