Optimization Problems for Machine Learning: A Survey

2019/01/16 by Claudio Gambella, Bissan Ghaddar, Joe Naoum-Sawaya +1 · 2 voices · 2 citations
Computer Science · Mathematics · #Anomaly Detection Techniques and Applications #Machine Learning and Algorithms #Machine Learning and Data Classification #cs.LG #math.OC

paper · pdf · doi:10.1016/j.ejor.2020.08.045

published as European Journal of Operational Research, 290, May 2021

openalex publication_date 2020/08/29 · arxiv created 2021/01/11 · arxiv updated 2021/01/12 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/29

Abstract

This paper surveys the machine learning literature and presents in an optimization framework several commonly used machine learning approaches. Particularly, mathematical optimization models are presented for regression, classification, clustering, deep learning, and adversarial learning, as well as new emerging applications in machine teaching, empirical model learning, and Bayesian network structure learning. Such models can benefit from the advancement of numerical optimization techniques which have already played a distinctive role in several machine learning settings. The strengths and the shortcomings of these models are discussed and potential research directions and open problems are highlighted.

Citations

• Adam: A Method for Stochastic Optimization
• Intriguing properties of neural networks
• Generalized Additive Model Selection
• BinaryConnect: Training Deep Neural Networks with binary weights during propagations
• A tutorial on support vector regression
• A Stochastic Approximation Method
• Cluster analysis and mathematical programming
• Sparse Approximate Solutions to Linear Systems
• Least angle regression
• Machine Teaching for Bayesian Learners in the Exponential Family
• Poisoning Attacks against Support Vector Machines
• Advances in Learning Bayesian Networks of Bounded Treewidth
• ImageNet classification with deep convolutional neural networks
• Activation Ensembles for Deep Neural Networks
• An Overview of Machine Teaching
• Rank-one Convexification for Sparse Regression
• A Survey on Neural Architecture Search
• Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizon
• On Assessing ML Model Robustness: A Methodological Framework (Academic Track)
• Generative Adversarial Nets
• An Introduction to Statistical Learning

Cited by

• A model-agnostic and data-independent tabu search algorithm to generate counterfactuals for tabular, image, and text data
• Machine learning at the service of meta-heuristics for solving combinatorial optimization problems: A state-of-the-art

Discussions

• Optimization Models for Machine Learning: A Survey (2019) [hn, 4 points, 0 comments]
• Optimization Problems for Machine Learning: A Survey [hn, 1 points, 0 comments]

Related

• A Survey of Optimization Methods from a Machine Learning Perspective
• Optimization Methods for Large-Scale Machine Learning
• Optimization Methods for Supervised Machine Learning: From Linear Models to Deep Learning
• Bayesian Optimization for Machine Learning : A Practical Guidebook
• Lecture Notes: Optimization for Machine Learning