Clustering Tails in High Dimension

2025/06/24 by Chen, Liujun, Oesting, Marco, Zhou, Chen
#FOS: Computer and information sciences #Methodology (stat.ME)

paper · doi:10.48550/arxiv.2506.19414

Abstract

One potential solution to combat the scarcity of tail observations in extreme value analysis is to integrate information from multiple datasets sharing similar tail properties, for instance, a common extreme value index. In other words, for a multivariate dataset, we intend to group dimensions into clusters first, before applying any pooling techniques. This paper addresses the clustering problem for a high dimensional dataset, according to their extreme value indices. We propose an iterative clustering procedure that sequentially partitions the variables into groups, ordered from the heaviest-tailed to the lightesttailed distributions. At each step, our method identifies and extracts a group of variables that share the highest extreme value index among the remaining ones. This approach differs fundamentally from conventional clustering methods such as using pre-estimated extreme value indices in a two-step clustering method. We show the consistency property of the proposed algorithm and demonstrate its finite-sample performance using a simulation study and a real data application.

Citations

• Representation Retrieval Learning for Heterogeneous Data Integration
• High dimensional inference for extreme value indices
• Multi-dimensional domain generalization with low-rank structures
• Graphical lasso for extremes
• Extremes in High Dimensions: Methods and Scalable Algorithms
• Learning extremal graphical structures in high dimensions
• Distributed Inference for Tail Risk
• Transfer Learning under High-dimensional Generalized Linear Models
• Structure learning for extremal tree models
• Transfer Learning for High-dimensional Linear Regression: Prediction, Estimation, and Minimax Optimality
• Spatial dependence and space-time trend in extreme events
• Distributed inference for quantile regression processes

Related

• k-means clustering of extremes
• Multivariate sparse clustering for extremes
• On estimation and order selection for multivariate extremes via clustering
• Clustering of multivariate tail dependence using conditional methods
• Clustering by the Probability Distributions from Extreme Value Theory