Data-driven confidence bands for distributed nonparametric regression

2019/12/13 by Valeriy Avanesov, Avanesov, Valeriy
Computer Science · Mathematics · #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Statistics Theory (math.ST) #cs.LG #math.ST #stat.ML #stat.TH

paper · pdf · doi:10.48550/arxiv.1912.06689

COLT2020 (to appear)

arxiv created 2020/06/08 · arxiv updated 2020/06/11

Abstract

Gaussian Process Regression and Kernel Ridge Regression are popular nonparametric regression approaches. Unfortunately, they suffer from high computational complexity rendering them inapplicable to the modern massive datasets. To that end a number of approximations have been suggested, some of them allowing for a distributed implementation. One of them is the divide and conquer approach, splitting the data into a number of partitions, obtaining the local estimates and finally averaging them. In this paper we suggest a novel computationally efficient fully data-driven algorithm, quantifying uncertainty of this method, yielding frequentist L₂-confidence bands. We rigorously demonstrate validity of the algorithm. Another contribution of the paper is a minimax-optimal high-probability bound for the averaged estimator, complementing and generalizing the known risk bounds.

Related

• Uncertainty quantification for distributed regression
• Optimal recovery and uncertainty quantification for distributed Gaussian process regression
• Divide and Conquer Kernel Ridge Regression: A Distributed Algorithm with Minimax Optimal Rates
• Uniform confidence bands for nonparametric errors-in-variables regression
• Practical Global and Local Bounds in Gaussian Process Regression via Chaining