2017/01/18 by Burak C. Civek, Civek, Burak C., İbrahim Delibalta +3
Computer Science · Decision Sciences · Engineering · #Advanced Adaptive Filtering Techniques #Advanced Bandit Algorithms Research #Control Systems and Identification #Data Stream Mining Techniques #FOS: Computer and information sciences #Machine Learning (cs.LG) #Neural Networks and Applications
paper · pdf · doi:10.48550/arxiv.1701.05053
openalex publication_date 2017/01/18 · openalex created_date 2022/08/15 · openalex updated_date 2026/07/28
We introduce highly efficient online nonlinear regression algorithms that are\nsuitable for real life applications. We process the data in a truly online\nmanner such that no storage is needed, i.e., the data is discarded after being\nused. For nonlinear modeling we use a hierarchical piecewise linear approach\nbased on the notion of decision trees where the space of the regressor vectors\nis adaptively partitioned based on the performance. As the first time in the\nliterature, we learn both the piecewise linear partitioning of the regressor\nspace as well as the linear models in each region using highly effective second\norder methods, i.e., Newton-Raphson Methods. Hence, we avoid the well known\nover fitting issues by using piecewise linear models, however, since both the\nregion boundaries as well as the linear models in each region are trained using\nthe second order methods, we achieve substantial performance compared to the\nstate of the art. We demonstrate our gains over the well known benchmark data\nsets and provide performance results in an individual sequence manner\nguaranteed to hold without any statistical assumptions. Hence, the introduced\nalgorithms address computational complexity issues widely encountered in real\nlife applications while providing superior guaranteed performance in a strong\ndeterministic sense.\n