The noise level in linear regression with dependent data

2023/05/18 by Ziemann, Ingvar, Tu, Stephen, Pappas, George J. +1
#FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Statistics Theory (math.ST)

paper · doi:10.48550/arxiv.2305.11165

Abstract

We derive upper bounds for random design linear regression with dependent (β-mixing) data absent any realizability assumptions. In contrast to the strictly realizable martingale noise regime, no sharp instance-optimal non-asymptotics are available in the literature. Up to constant factors, our analysis correctly recovers the variance term predicted by the Central Limit Theorem -- the noise level of the problem -- and thus exhibits graceful degradation as we introduce misspecification. Past a burn-in, our result is sharp in the moderate deviations regime, and in particular does not inflate the leading order term by mixing time factors.

Related

• A concentration inequality for the excess risk in least-squares regression with random design and heteroscedastic noise
• Noisy recovery from random linear observations: Sharp minimax rates under elliptical constraints
• On deviation probabilities in non-parametric regression with heavy-tailed noise
• Nonparametric regression estimation for random fields in a fixed-design
• Generalization bounds for nonparametric regression with β-mixing samples