2023/10/13 by Whitehouse, Justin, Wu, Zhiwei Steven, Ramdas, Aaditya · 5 citations
#FOS: Computer and information sciences #FOS: Mathematics #Methodology (stat.ME) #Probability (math.PR) #Statistics Theory (math.ST)
paper · doi:10.48550/arxiv.2310.09100
Self-normalized processes arise naturally in many learning-related tasks. While self-normalized concentration has been extensively studied for scalar-valued processes, there are few results for multidimensional processes outside of the sub-Gaussian setting. In this work, we construct a general, self-normalized inequality for multivariate processes that satisfy a simple yet broad sub-ψ tail condition, which generalizes assumptions based on cumulant generating functions. From this general inequality, we derive an upper law of the iterated logarithm for sub-ψ vector-valued processes, which is tight up to small constants. We show how our inequality can be leveraged to derive a variety of novel, self-normalized concentration inequalities under both light and heavy-tailed observations. Further, we provide applications in prototypical statistical tasks, such as parameter estimation in online linear regression, autoregressive modeling, and bounded mean estimation via a new (multivariate) empirical Bernstein concentration inequality.