On Learning Prediction-Focused Mixtures

2021/10/25 by Abhishek Sharma, Catherine Zeng, Sharma, Abhishek +7
Computer Science · #Artificial Intelligence (cs.AI) #Data Stream Mining Techniques #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning in Healthcare

paper · pdf · doi:10.48550/arxiv.2110.13221

openalex publication_date 2021/10/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Probabilistic models help us encode latent structures that both model the data and are ideally also useful for specific downstream tasks. Among these, mixture models and their time-series counterparts, hidden Markov models, identify discrete components in the data. In this work, we focus on a constrained capacity setting, where we want to learn a model with relatively few components (e.g. for interpretability purposes). To maintain prediction performance, we introduce prediction-focused modeling for mixtures, which automatically selects the dimensions relevant to the prediction task. Our approach identifies relevant signal from the input, outperforms models that are not prediction-focused, and is easy to optimize; we also characterize when prediction-focused modeling can be expected to work.

Citations

• The information bottleneck method
• Prediction-Constrained Training for Semi-Supervised Mixture and Topic\n Models

Related

• Infinite Mixture Model of Markov Chains
• On Learning Mixture of Linear Regressions in the Non-Realizable Setting
• Prediction Sets for High-Dimensional Mixture of Experts Models
• Learning Mixture Structure on Multi-Source Time Series for Probabilistic Forecasting
• Mixture Weight Estimation and Model Prediction in Multi-source Multi-target Domain Adaptation