2015/06/30 by Nicholas Chancellor, Szilárd Szőke, Szilard Szoke +5 · 1 citation
Computer Science · Mathematics · Physics and Astronomy · #Algorithm #Artificial intelligence #Bayes' theorem #Bayesian probability #Boltzmann machine #Computer science #Decoding methods #Deep learning #Entropy (arrow of time) #Inference #Ising model #Mathematics #Maximum a posteriori estimation #Maximum likelihood #Neural Networks and Applications #Neural Networks and Reservoir Computing #Physics #Principle of maximum entropy #Quantum Computing Algorithms and Architecture #Quantum mechanics #Statistical physics #Statistics #physics.comp-ph #quant-ph
paper · pdf · doi:10.1038/srep22318
published as Scientific Reports 6, Article number: 22318 (2016) · 9 figures in main text 9 figures in supplemental material. Significant amount of new Monte Carlo data added in v2 at referees request. Accepted for Scientific Reports
arxiv created 2016/03/01 · openalex publication_date 2016/03/03 · arxiv updated 2016/03/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05
Optimisation problems typically involve finding the ground state (i.e. the minimum energy configuration) of a cost function with respect to many variables. If the variables are corrupted by noise then this maximises the likelihood that the solution is correct. The maximum entropy solution on the other hand takes the form of a Boltzmann distribution over the ground and excited states of the cost function to correct for noise. Here we use a programmable annealer for the information decoding problem which we simulate as a random Ising model in a field. We show experimentally that finite temperature maximum entropy decoding can give slightly better bit-error-rates than the maximum likelihood approach, confirming that useful information can be extracted from the excited states of the annealer. Furthermore we introduce a bit-by-bit analytical method which is agnostic to the specific application and use it to show that the annealer samples from a highly Boltzmann-like distribution. Machines of this kind are therefore candidates for use in a variety of machine learning applications which exploit maximum entropy inference, including language processing and image recognition.