PMODE: Theoretically Grounded and Modular Mixture Modeling

2025/08/29 by Robert A. Vandermeulen, Vandermeulen, Robert A.
Computer Science · #Anomaly Detection Techniques and Applications #Bayesian Methods and Mixture Models #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · pdf · doi:10.48550/arxiv.2508.21396

openalex publication_date 2025/08/29 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We introduce PMODE (Partitioned Mixture Of Density Estimators), a general and modular framework for mixture modeling with both parametric and nonparametric components. PMODE builds mixtures by partitioning the data and fitting separate estimators to each subset. It attains near-optimal rates for this estimator class and remains valid even when the mixture components come from different distribution families. As an application, we develop MV-PMODE, which scales a previously theoretical approach to high-dimensional density estimation to settings with thousands of dimensions. Despite its simplicity, it performs competitively against deep baselines on CIFAR-10 anomaly detection.

Citations

• Dimension-independent rates for structured neural density estimation
• Breaking the curse of dimensionality in structured density estimation
• Generalized multi-view model: Adaptive density estimation under low-rank constraints
• Sample Complexity Using Infinite Multiview Models
• An Equal-Size Hard EM Algorithm for Diverse Dialogue Generation
• Generalized Identifiability Bounds for Mixture Models with Grouped Samples
• Identifiability of deep generative models without auxiliary information
• Exposing Outlier Exposure: What Can Be Learned From Few, One, and Zero Outlier Images
• Beyond Smoothness: Incorporating Low-Rank Analysis into Nonparametric Density Estimation
• FastFlow: Unsupervised Anomaly Detection and Localization via 2D Normalizing Flows
• Recovery Guarantees for Kernel-based Clustering under Non-parametric Mixture Models
• Uniform Consistency in Nonparametric Mixture Models
• A Unifying Review of Deep and Shallow Anomaly Detection
• CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances
• Array Programming with NumPy
• Consistent Estimation of Identifiable Nonparametric Mixture Models from Grouped Observations
• Estimation of the Number of Components of Non-Parametric Multivariate Finite Mixture Models
• SciPy 1.0--Fundamental Algorithms for Scientific Computing in Python
• Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty
• Learning Mixtures of Smooth Product Distributions: Identifiability and Algorithm
• Deep Anomaly Detection with Outlier Exposure
• Do Deep Generative Models Know What They Don't Know?
• Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learning
• Deep Anomaly Detection Using Geometric Transformations
• Identifiability of Nonparametric Mixture Models and Bayes Optimal Clustering
• Deep Gaussian Mixture Models
• Deep Gaussian mixture models
• On w-mixtures: Finite convex combinations of prescribed component distributions
• Sample-Efficient Learning of Mixtures
• Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery
• Nonparametric Estimation of Multi-View Latent Variable Models
• The More, the Merrier: the Blessing of Dimensionality for Learning Large Gaussian Mixtures
• A Practical Algorithm for Topic Modeling with Provable Guarantees
• Learning Topic Models - Going beyond SVD
• Robust Kernel Density Estimation
• Density estimation in linear time
• On the Identifiability of Finite Mixtures
• Tight Bounds on the Hardness of Learning Simple Nonparametric Mixtures

Related

• Staged Mixture Modelling and Boosting
• MixEst: An Estimation Toolbox for Mixture Models
• From Adaptive Kernel Density Estimation to Sparse Mixture Models
• Mixture Proportion Estimation via Kernel Embedding of Distributions
• Bayesian Density Estimation and Inference Using Mixtures