Technical report: Training Mixture Density Networks with full covariance matrices

2020/03/04 by Jakob Kruse, Kruse, Jakob
Computer Science · #Bayesian Methods and Mixture Models #Bayesian Modeling and Causal Inference #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG)

paper · pdf · doi:10.48550/arxiv.2003.05739

openalex publication_date 2020/03/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Mixture Density Networks are a tried and tested tool for modelling conditional probability distributions. As such, they constitute a great baseline for novel approaches to this problem. In the standard formulation, an MDN takes some input and outputs parameters for a Gaussian mixture model with restrictions on the mixture components' covariance. Since covariance between random variables is a central issue in the conditional modeling problems we were investigating, I derived and implemented an MDN formulation with unrestricted covariances. It is likely that this has been done before, but I could not find any resources online. For this reason, I have documented my approach in the form of this technical report, in hopes that it may be useful to others facing a similar situation.

Related

• A BGe score for tied-covariance mixture Gaussian Bayesian networks
• Easy Conditioning far beyond Gaussian
• Mixture Density Conditional Generative Adversarial Network Models\n (MD-CGAN)
• Mixture Density Networks for Classification with an Application to Product Bundling
• Mix-nets: Factored Mixtures of Gaussians in Bayesian Networks With Mixed Continuous And Discrete Variables