2005/08/31 by Kazushi Mimura, Masato Okada · 1 citation
Computer Science · Mathematics · Physics and Astronomy · #Algorithm #Algorithms and Data Compression #Artificial intelligence #Artificial neural network #Combinatorics #Compression (physics) #Computer science #Data compression #Discrete mathematics #Error Correcting Code Techniques #Lossy compression #Mathematical analysis #Mathematics #Monotonic function #Neural Networks and Applications #Parity (physics) #Perceptron #Physics #Quantum mechanics #Statistical mechanics #Statistical physics #Tree (set theory) #Upper and lower bounds #cond-mat.dis-nn #cond-mat.stat-mech
paper · pdf · doi:10.1103/physreve.74.026108
published as Phys. Rev. E, 74, 026108 (2006) · 12 pages, 5 figures
arxiv created 2006/05/02 · openalex publication_date 2006/08/08 · arxiv updated 2009/12/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05
Statistical mechanics is applied to lossy compression using multilayer perceptrons for unbiased Boolean messages. We utilize a treelike committee machine (committee tree) and treelike parity machine (parity tree) whose transfer functions are monotonic. For compression using a committee tree, a lower bound of achievable distortion becomes small as the number of hidden units K increases. However, it cannot reach the Shannon bound even where K-->infinity. For a compression using a parity tree with K> or =2 hidden units, the rate distortion function, which is known as the theoretical limit for compression, is derived where the code length becomes infinity.