2019/07/15 by Jean Barbier, Barbier, Jean
Computer Science · Engineering · Mathematics · Physics and Astronomy · #FOS: Computer and information sciences #FOS: Electrical engineering #FOS: Mathematics #Information Theory (cs.IT) #Machine Learning (cs.LG) #Model Reduction and Neural Networks #Neural Networks and Applications #Probability (math.PR) #Signal Processing (eess.SP) #Stochastic Gradient Optimization Techniques #cs.IT #cs.LG #eess.SP #electronic engineering #information engineering #math.IT #math.PR
paper · pdf · doi:10.48550/arxiv.1907.07103
arXiv admin note: text overlap with arXiv:1904.02808
arxiv created 2019/07/15 · openalex publication_date 2019/07/15 · arxiv updated 2019/07/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We consider Bayesian inference of signals with vector-valued entries. Extending concentration techniques from the mathematical physics of spin glasses, we show that the matrix-valued minimum mean-square error concentrates when the size of the problem increases. Such results are often crucial for proving single-letter formulas for the mutual information when they exist. Our proof is valid in the optimal Bayesian inference setting, meaning that it relies on the assumption that the model and all its hyper-parameters are known. Examples of inference and learning problems covered by our results are spiked matrix and tensor models, the committee machine neural network with few hidden neurons in the teacher-student scenario, or multi-layers generalized linear models.