2022/06/20 by Ruqi Zhang, Xingchao Liu, Zhang, Ruqi +3 · 1 voice · 11 citations
Computer Science · Mathematics · #Algorithm #Applied mathematics #Artificial intelligence #Artificial neural network #Bayesian Methods and Mixture Models #Binary number #Boltzmann distribution #Boltzmann machine #Computer science #Gaussian Processes and Bayesian Inference #Ising model #Langevin dynamics #Markov Chains and Monte Carlo Methods #Mathematics #Physics #Quadratic equation #Sampling (signal processing) #Scalability #Statistical physics #cs.LG #stat.ML
paper · pdf · doi:10.48550/arxiv.2206.09914
published in arXiv (Cornell University) (Cornell University) · Published at ICML 2022
arxiv created 2022/06/20 · openalex publication_date 2022/06/20 · arxiv updated 2022/06/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05
We propose discrete Langevin proposal (DLP), a simple and scalable gradient-based proposal for sampling complex high-dimensional discrete distributions. In contrast to Gibbs sampling-based methods, DLP is able to update all coordinates in parallel in a single step and the magnitude of changes is controlled by a stepsize. This allows a cheap and efficient exploration in the space of high-dimensional and strongly correlated variables. We prove the efficiency of DLP by showing that the asymptotic bias of its stationary distribution is zero for log-quadratic distributions, and is small for distributions that are close to being log-quadratic. With DLP, we develop several variants of sampling algorithms, including unadjusted, Metropolis-adjusted, stochastic and preconditioned versions. DLP outperforms many popular alternatives on a wide variety of tasks, including Ising models, restricted Boltzmann machines, deep energy-based models, binary neural networks and language generation.