2023/05/26 by Fabio A. González, Raúl Ramos-Pollán, González, Fabio A. +3 · 2 citations
Computer Science · Physics and Astronomy · #FOS: Computer and information sciences #FOS: Physical sciences #Gaussian Processes and Bayesian Inference #I.2.6 #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Neural Networks and Applications #Quantum Physics (quant-ph) #Statistical Mechanics and Entropy
paper · pdf · doi:10.48550/arxiv.2305.18204
openalex publication_date 2023/05/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
This paper introduces a novel approach to probabilistic deep learning, kernel density matrices, which provide a simpler yet effective mechanism for representing joint probability distributions of both continuous and discrete random variables. In quantum mechanics, a density matrix is the most general way to describe the state of a quantum system. This work extends the concept of density matrices by allowing them to be defined in a reproducing kernel Hilbert space. This abstraction allows the construction of differentiable models for density estimation, inference, and sampling, and enables their integration into end-to-end deep neural models. In doing so, we provide a versatile representation of marginal and joint probability distributions that allows us to develop a differentiable, compositional, and reversible inference procedure that covers a wide range of machine learning tasks, including density estimation, discriminative learning, and generative modeling. The broad applicability of the framework is illustrated by two examples: an image classification model that can be naturally transformed into a conditional generative model, and a model for learning with label proportions that demonstrates the framework's ability to deal with uncertainty in the training samples. The framework is implemented as a library and is available at: https://github.com/fagonzalezo/kdm.