Iterative proportional scaling revisited: a modern optimization perspective

2016/10/08 by She, Yiyuan, Tang, Shao
#Computation (stat.CO) #FOS: Computer and information sciences #Machine Learning (stat.ML)

paper · doi:10.48550/arxiv.1610.02588

Abstract

This paper revisits the classic iterative proportional scaling (IPS) from a modern optimization perspective. In contrast to the criticisms made in the literature, we show that based on a coordinate descent characterization, IPS can be slightly modified to deliver coefficient estimates, and from a majorization-minimization standpoint, IPS can be extended to handle log-affine models with features not necessarily binary-valued or nonnegative. Furthermore, some state-of-the-art optimization techniques such as block-wise computation, randomization and momentum-based acceleration can be employed to provide more scalable IPS algorithms, as well as some regularized variants of IPS for concurrent feature selection.

Related

• On the effects of scaling on the performance of Ipopt
• Scaling Exponents Across Parameterizations and Optimizers
• POCAII: Parameter Optimization with Conscious Allocation using Iterative Intelligence
• Enhancing Optimizer Stability: Momentum Adaptation of The NGN Step-size
• Parallel black-box optimization of expensive high-dimensional multimodal functions via magnitude