Free energy landscape of Dense Associative Memory

2026/07/21 by Sumedha, Abhishek Singh
#cond-mat.dis-nn #cond-mat.stat-mech #cs.AI

paper · pdf

Abstract

Using large deviations theory, we solve and obtain a general expression for the free energy functional for a broad class of associative memories, including dense associative memories. We illustrate the method by reproducing classical results for the Hopfield model. For a finite number of patterns, we derive the temperature-dependent free energy functional for dense associative memories featuring polynomial interactions and Log-Sum-Exponential (LSE) activation. We also evaluate the disorder-averaged ground-state energy of these systems in the extensive limit. Our analytical framework reveals how memory retrieval depends on the initial state in higher-order dense networks, and gives the exact full-retrieval threshold for the LSE model. This method provides a systematic procedure for analyzing diverse, complex architectures in associative memory.

Citations

• There Will Be a Scientific Theory of Deep Learning
• Hopfield Networks is All You Need
• On a Model of Associative Memory with Huge Storage Capacity

Related

• Dynamical Properties of Dense Associative Memory
• Dense Associative Memory with Epanechnikov Energy
• Dense Associative Memory Through the Lens of Random Features
• Enhancing associative memory recall in non-equilibrium materials through activity
• Large Associative Memory Problem in Neurobiology and Machine Learning