2020/03/02 by Jacob Miller, Miller, Jacob, Guillaume Rabusseau +3 · 1 citation
Computer Science · Mathematics · Physics and Astronomy · #FOS: Computer and information sciences #FOS: Physical sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Parallel Computing and Optimization Techniques #Quantum Physics (quant-ph) #Quantum many-body systems #Tensor decomposition and applications #cs.LG #quant-ph #stat.ML
paper · pdf · doi:10.48550/arxiv.2003.01039
18 pages, 2 figures; v4 conference version; v3 link to code for experiments; v2 major revision with new main result on regular expression sampling. International Conference on Artificial Intelligence and Statistics. PMLR, 2021
openalex publication_date 2020/03/02 · arxiv created 2021/04/23 · arxiv updated 2021/04/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Tensor networks are a powerful modeling framework developed for computational many-body physics, which have only recently been applied within machine learning. In this work we utilize a uniform matrix product state (u-MPS) model for probabilistic modeling of sequence data. We first show that u-MPS enable sequence-level parallelism, with length-n sequences able to be evaluated in depth O(log n). We then introduce a novel generative algorithm giving trained u-MPS the ability to efficiently sample from a wide variety of conditional distributions, each one defined by a regular expression. Special cases of this algorithm correspond to autoregressive and fill-in-the-blank sampling, but more complex regular expressions permit the generation of richly structured data in a manner that has no direct analogue in neural generative models. Experiments on sequence modeling with synthetic and real text data show u-MPS outperforming a variety of baselines and effectively generalizing their predictions in the presence of limited data.