2017/03/15 by Elena Agliari, Adriano Barra, Chiara Longo +1
Computer Science · Mathematics · Physics and Astronomy · #Artificial neural network #Boltzmann machine #Boolean function #Computation #Content-addressable memory #Function (biology) #Hopfield network #Neural Networks and Applications #Quantum many-body systems #Replica #Restricted Boltzmann machine #Stability (learning theory) #Stochastic Gradient Optimization Techniques #cond-mat.dis-nn #math-ph #math.MP
paper · pdf · doi:10.1007/s10955-017-1840-9
16 pages, 1 figure
arxiv created 2017/03/15 · openalex created_date 2017/03/23 · openalex publication_date 2017/07/17 · arxiv updated 2017/08/02 · openalex updated_date 2026/08/05
Restricted Boltzmann Machines are key tools in Machine Learning and are described by the energy function of bipartite spin-glasses. From a statistical mechanical perspective, they share the same Gibbs measure of Hopfield networks for associative memory. In this equivalence, weights in the former play as patterns in the latter. As Boltzmann machines usually require real weights to be trained with gradient descent like methods, while Hopfield networks typically store binary patterns to be able to retrieve, the investigation of a mixed Hebbian network, equipped with both real (e.g., Gaussian) and discrete (e.g., Boolean) patterns naturally arises. We prove that, in the challenging regime of a high storage of real patterns, where retrieval is forbidden, an extra load of Boolean patterns can still be retrieved, as long as the ratio among the overall load and the network size does not exceed a critical threshold, that turns out to be the same of the standard Amit-Gutfreund-Sompolinsky theory. Assuming replica symmetry, we study the case of a low load of Boolean patterns combining the stochastic stability and Hamilton-Jacobi interpolating techniques. The result can be extended to the high load by a non rigorous but standard replica computation argument.